Loading Additional Ruby Gems in Development Using your favorite gems without adding them to your project’s Gemfile

October 20, 2019

Have you ever found yourself temporarily adding a gem to your project’s Gemfile to leverage it during development? Maybe your project even has gems that some team members find indispensable during development, but mean nothing to others. Luckily, there’s a way to have your favorite gems available without permanently or temporarily adding them to the project’s Gemfile.

Recently at my company we were spring cleaning our app’s Gemfile. Over the years people have added gems to it that they heavily rely on during development, myself included.

I always use the Fuubar formatter for RSpec and greatly prefer it over the default formatter. Sporadically I might need to get a flame graph for a performance critical part of the app, so I temporarily add the Flamegraph gem. Other times I can’t wrap my head around Rails’ implicit i18n key resolution, so I quickly add my i18n-debug gem.

While doing this spring cleaning someone pointed out that there’s a way to use a different Gemfile locally that allows you to add additional gems. This sounded like the correct approach that would make everyone happy.

As it wasn’t straightforward to set this up and getting it to work correctly, especially in the context of a Rails app using Spring and Puma-dev or Pow, I wanted to document the necessary steps. Hopefully this comes in handy for someone else.

Setting up a Private Gemfile

In the root directory of a typical Ruby project you’ll have a Gemfile and a Gemfile.lock. The first step is creating a second Gemfile with a name of your choice, e.g. Gemfile.private. I first went with Gemfile.local as that’s what you find here and there on the internet, but I’ve found that Gemfile.local, Gemfile.lock, and Gemfile.local.lock are all too similar and ambiguous when using tab completion.

This file will evaluate the project’s Gemfile and list additional gems of your choice. Here’s how mine looks like:

eval_gemfile 'Gemfile'

gem 'fuubar'

gem 'stackprof'
gem 'flamegraph'

gem 'i18n-debug'

We now need to run bundle install while instructing Bundler to use this Gemfile instead.

Before doing so, it makes sense to use the Gemfile.lock as the starting point for version resolution. This ensures that we’ll get the same versions of the gems defined in Gemfile, especially if you’re using pessimistic version constraints.

$ cp Gemfile.lock Gemfile.private.lock

Now we can bundle install using Gemfile.private:

$ BUNDLE_GEMFILE=Gemfile.private bundle install

Because you’ll be using Gemfile.private during development, you’ll have to remember to do changes, which should be checked in, directly on the Gemfile and Gemfile.lock, e.g. when updating or adding a gem.

On the other hand, whenever Gemfile.lock was changed upstream, you’ll have to redo the copy and bundle install step for your private Gemfile in order for your Gemfile.private.lock to include those upstream changes.

I have written a Bundler plugin that takes care of this step whenever running bundle install. To install the plugin globally, run the following in a directory that doesn’t have a Gemfile:

$ bundle plugin install bundler-private_install

Executing Commands Using Your Private Gemfile

When running a command through Bundler, you’ll need to set the BUNDLE_GEMFILE environment variable for Bundler to load the gems specified in your private Gemfile.

$ BUNDLE_GEMFILE=Gemfile.private bundle exec foobar

You can also export BUNDLE_GEMFILE in your shell so you don’t have to repeat it for every command:

$ export BUNDLE_GEMFILE=Gemfile.private
$ bundle exec foobar

If you’re using Bundler binstubs, have a look at a binstub. You’ll notice at the top of a binstub that it supports BUNDLE_GEMFILE:

ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",

This means that your binstubs will pick up your private Gemfile:

$ BUNDLE_GEMFILE=Gemfile.private bin/rake

Rails binstubs are slightly different as they don’t load Bundler directly, but instead load config/boot.rb, which supports BUNDLE_GEMFILE as well. At the top of config/boot.rb you’ll see a similar snippet:

ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../Gemfile', __dir__)

A few lines later, after having set BUNDLE_GEMFILE, it loads the Bundler setup.

If you’re working with Spring and a server such as Puma-dev or Pow, you’ll have to go through a few extra hoops described below.

Configuring Spring

When running commands that leverage a preloaded application through Spring, such as rails console or rspec, it will not pick up the BUNDLE_GEMFILE that’s set in the shell.

Luckily, Spring provides configuration hooks:

Spring will read ~/.spring.rb and config/spring.rb for custom settings. Note that ~/.spring.rb is loaded before bundler, but config/spring.rb is loaded after bundler.

Because we need BUNDLE_GEMFILE to be set before Bundler is loaded and because we want to do this config globally and not for every project where we want a private Gemfile, the ~/.spring.rb file is the right spot.

Add the following snippet to ~/.spring.rb that sets BUNDLE_GEMFILE if there is one (note that this file gets evaluated in your project’s root):

ENV['BUNDLE_GEMFILE'] = 'Gemfile.private' if File.exist?('Gemfile.private')

Now, whenever you’re working on an app that has a Gemfile.private, you’ll have your additional gems available inside Spring-powered commands.

Configuring Puma-dev or Pow

Similarly as for Spring, we need to configure Puma-dev or Pow to set BUNDLE_GEMFILE.

Puma-dev also features a set of config files according to the advanced configuration section of the docs:

Puma-dev supports loading environment variables before puma starts. It checks for the following files in this order:

  • ~/.powconfig
  • .env
  • .powrc
  • .powenv

The ~/.powconfig file is exactly what we need. Puma-dev, dubbed “the emotional successor to pow”, simply uses the same file as Pow. In case you’re still using Pow, the relevant section of the manual is here.

Analogous to ~/.spring.rb, we set BUNDLE_GEMFILE in ~/.powconfig, which is a shell script:

[ -f Gemfile.private ] && export BUNDLE_GEMFILE=Gemfile.private

As for Spring, this file is evaluated in the root of the app being booted.

Remember to restart the app with touch tmp/restart.txt if it was running. After that, whenever Puma-dev boots an app, it’ll first set BUNDLE_GEMFILE if a private Gemfile exists.

I hope that this guide helped you in setting up your private Gemfile.

As a final tip, if you’re working on multiple projects and find yourself adding the same set of gems, you could have a global private Gemfile, e.g. at ~/Gemfile.private and simply link to that file in your project:

$ ln -s ~/Gemfile.private