Loading Additional Ruby Gems in Development Using your favorite gems without adding them to your project’s Gemfile
Have you ever found yourself temporarily adding a gem to your project’s Gemfile to leverage it during development? Maybe your project even has gems that some team members find indispensable during development, but mean nothing to others. Luckily, there’s a way to have your favorite gems available without permanently or temporarily adding them to the project’s Gemfile.
Recently at my company we were spring cleaning our app’s Gemfile. Over the years people have added gems to it that they heavily rely on during development, myself included.
I always use the Fuubar formatter for RSpec and greatly prefer it over the default formatter. Sporadically I might need to get a flame graph for a performance critical part of the app, so I temporarily add the Flamegraph gem. Other times I can’t wrap my head around Rails’ implicit i18n key resolution, so I quickly add my i18n-debug gem.
While doing this spring cleaning someone pointed out that there’s a way to use a different Gemfile locally that allows you to add additional gems. This sounded like the correct approach that would make everyone happy.
As it wasn’t straightforward to set this up and getting it to work correctly, especially in the context of a Rails app using Spring and Puma-dev or Pow, I wanted to document the necessary steps. Hopefully this comes in handy for someone else.
- Setting up a Private Gemfile
- Executing Commands Using Your Private Gemfile
- Configuring Spring
- Configuring Puma-dev or Pow
Setting up a Private Gemfile
In the root directory of a typical Ruby project you’ll have a Gemfile
and a Gemfile.lock
. The first step is creating a second Gemfile with a name of your choice, e.g. Gemfile.private
. I first went with Gemfile.local
as that’s what you find here and there on the internet, but I’ve found that Gemfile.local
, Gemfile.lock
, and Gemfile.local.lock
are all too similar and ambiguous when using tab completion.
This file will evaluate the project’s Gemfile
and list additional gems of your choice. Here’s how mine looks like:
eval_gemfile 'Gemfile'
gem 'fuubar'
gem 'stackprof'
gem 'flamegraph'
gem 'i18n-debug'
We now need to run bundle install
while instructing Bundler to use this Gemfile instead.
Before doing so, it makes sense to use the Gemfile.lock
as the starting point for version resolution. This ensures that we’ll get the same versions of the gems defined in Gemfile
, especially if you’re using pessimistic version constraints.
$ cp Gemfile.lock Gemfile.private.lock
Now we can bundle install
using Gemfile.private
:
$ BUNDLE_GEMFILE=Gemfile.private bundle install
Because you’ll be using Gemfile.private
during development, you’ll have to remember to do changes, which should be checked in, directly on the Gemfile
and Gemfile.lock
, e.g. when updating or adding a gem.
On the other hand, whenever Gemfile.lock
was changed upstream, you’ll have to redo the copy and bundle install
step for your private Gemfile in order for your Gemfile.private.lock
to include those upstream changes.
I have written a Bundler plugin that takes care of this step whenever running bundle install
. To install the plugin globally, run the following in a directory that doesn’t have a Gemfile:
$ bundle plugin install bundler-private_install
Executing Commands Using Your Private Gemfile
When running a command through Bundler, you’ll need to set the BUNDLE_GEMFILE
environment variable for Bundler to load the gems specified in your private Gemfile.
$ BUNDLE_GEMFILE=Gemfile.private bundle exec foobar
You can also export BUNDLE_GEMFILE
in your shell so you don’t have to repeat it for every command:
$ export BUNDLE_GEMFILE=Gemfile.private
$ bundle exec foobar
If you’re using Bundler binstubs, have a look at a binstub. You’ll notice at the top of a binstub that it supports BUNDLE_GEMFILE
:
ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
Pathname.new(__FILE__).realpath)
This means that your binstubs will pick up your private Gemfile:
$ BUNDLE_GEMFILE=Gemfile.private bin/rake
Rails binstubs are slightly different as they don’t load Bundler directly, but instead load config/boot.rb
, which supports BUNDLE_GEMFILE
as well. At the top of config/boot.rb
you’ll see a similar snippet:
ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../Gemfile', __dir__)
A few lines later, after having set BUNDLE_GEMFILE
, it loads the Bundler setup.
If you’re working with Spring and a server such as Puma-dev or Pow, you’ll have to go through a few extra hoops described below.
Configuring Spring
When running commands that leverage a preloaded application through Spring, such as rails console
or rspec
, it will not pick up the BUNDLE_GEMFILE
that’s set in the shell.
Luckily, Spring provides configuration hooks:
Spring will read
~/.spring.rb
andconfig/spring.rb
for custom settings. Note that~/.spring.rb
is loaded before bundler, butconfig/spring.rb
is loaded after bundler.
Because we need BUNDLE_GEMFILE
to be set before Bundler is loaded and because we want to do this config globally and not for every project where we want a private Gemfile, the ~/.spring.rb
file is the right spot.
Add the following snippet to ~/.spring.rb
that sets BUNDLE_GEMFILE
if there is one (note that this file gets evaluated in your project’s root):
ENV['BUNDLE_GEMFILE'] = 'Gemfile.private' if File.exist?('Gemfile.private')
Now, whenever you’re working on an app that has a Gemfile.private
, you’ll have your additional gems available inside Spring-powered commands.
Configuring Puma-dev or Pow
Similarly as for Spring, we need to configure Puma-dev or Pow to set BUNDLE_GEMFILE
.
Puma-dev also features a set of config files according to the advanced configuration section of the docs:
Puma-dev supports loading environment variables before puma starts. It checks for the following files in this order:
~/.powconfig
.env
.powrc
.powenv
The ~/.powconfig
file is exactly what we need. Puma-dev, dubbed “the emotional successor to pow”, simply uses the same file as Pow. In case you’re still using Pow, the relevant section of the manual is here.
Analogous to ~/.spring.rb
, we set BUNDLE_GEMFILE
in ~/.powconfig
, which is a shell script:
[ -f Gemfile.private ] && export BUNDLE_GEMFILE=Gemfile.private
As for Spring, this file is evaluated in the root of the app being booted.
Remember to restart the app with touch tmp/restart.txt
if it was running. After that, whenever Puma-dev boots an app, it’ll first set BUNDLE_GEMFILE
if a private Gemfile exists.
I hope that this guide helped you in setting up your private Gemfile.
As a final tip, if you’re working on multiple projects and find yourself adding the same set of gems, you could have a global private Gemfile, e.g. at ~/Gemfile.private
and simply link to that file in your project:
$ ln -s ~/Gemfile.private