Florian Munz, Co-Founder at Depfu

As soon as you start using private Github repos in your Gemfile you have to figure out how Bundler can access them. This is super easy on your local machine, as Bundler just uses your credentials, so if you can access the repo, so can Bundler.

Things get a bit trickier when you want to use a CI service like Travis or when you want to deploy your app to production.

Since we needed to support private dependencies in Depfu I looked at how to implement it and was actually surprised how many different ways there are. Let’s have a look:

Let’s transport some Git

There are 3 different transport mechanisms you can use for your Git repos:

GIT
gem 'rails', git: 'git://github.com/rails/rails.git'
SSH
gem 'rails', git: 'git@github.com:rails/rails.git'
HTTPS
gem 'rails', git: 'https://github.com/rails/rails.git'

To get this out of the way first: don’t use the git:// protocol. It’s insecure and allows a man-in-the-middle attack. Also, don’t use the github: shorthand, as it uses the Git protocol until Bundler 2.0.

That leaves us with HTTPS and SSH for specifying your private Git dependencies.

Using SSH

I think most people are familiar with how SSH works together with Bundler since there is a long history of using SSH: The SSH transport was the first one Github supported. Travis integrates with your code via SSH by default. And of course Capistrano is based on SSH.

In order for SSH to work with Bundler, the user executing bundle install needs to have an authorized private key for the Github repo. Github allows multiple keys to be associated with a single user account or with a single repo (deploy keys). Deploy keys sound great, but their use is complicated by the fact that GitHub does not allow you to reuse keys. So a single private key cannot access multiple Github repositories, which makes it quiet hard to use with private dependencies.

Since you usually don’t want to tie your CI or deploy to your own Github user account, a common pattern is to use a machine user: A newly created Github user that has read access to all the repos it needs access to, but it’s only used by your deploy scripts.

Using HTTPS

Git’s support for HTTPS with private repos is newer than SSH, but it has been around for quite some time now. Still, this is where I was the most surprised how useful it has become.

If you’re using HTTPS with private repos, Github requires you to supply your auth info using basic auth. By default, it asks for your username and password:

$ git clone https://github.com/username/repo.git
Username: your_username
Password: your_password

Instead of using your password, you can also use use personal access tokens. The advantage to using a token over your password is that a token can be revoked and you can define what permissions it gets. Also if you use 2FA on your Github account, you have to use tokens when using HTTPS.

So how can you let Bundler know about how to authenticate?

Manual every time

If you just put a private repo in your Gemfile, Bundler will actually ask you for the username/password every time you run bundle install. That’s quite annoying and also doesn’t really work on Travis or when deploying.

Directly in the Gemfile (don’t do this)

gem 'private', git: 'https://<username>:<password>@github.com/mycompany/private.git'
gem 'private', git: 'https://x-access-token:<token>@github.com/mycompany/private.git'

Adding it directly to the Gemfile makes clear what’s going on, but you’re committing sensitive information to your source code, which you really shouldn’t. And no, using a token here doesn’t really make it much better than username/password.

bundle config

You can configure Bundler with a few options, which it stores in a file. Bundler gets its configuration from the local app (app/.bundle/config), environment variables, and the user’s home directory (~/.bundle/config), in that order of priority.

You can add authentication information to it like this:

$ bundle config GITHUB__COM username:password
$ bundle config --local GITHUB__COM username:password

This takes your username/password out of the source code, but it’s stored in cleartext on your computer. Also you still have to tell your servers and Travis about this.

bundle config with environment variables

Bundler actually allows all config options to also be set via environment variables:

$ export BUNDLE_GITHUB__COM=username:password

Or if you’re using personal access tokens

$ export BUNDLE_GITHUB__COM=x-access-token:<token>

This technique works really well with Travis and Heroku, since they allow environment variables to be stored securely.

⚠️  Github also allows <token>:x-oauth-basic as the username/password for basic auth, but it’s not recommended anymore since not all tools are hiding the token in their output. So it could end up in your logfiles.

Caching your credentials

This is a less known technique that you can use locally on a Mac. Instead of Bundler (or Git) asking you every time for your username and password, you can store them in your macOS keychain. So you’ll be only asked the first time and all further checkouts will use the cached credentials.

It’s quite easy to set up and brings the HTTPS transport to the same level as SSH from a convenience point of view.

Putting it all together

In the past I usually relied on the SSH transport for everything: How we clone our main repo and how we specify private Git dependencies.

But I feel that HTTPS is on the same level now when using environment variables for CI/Heroku and caching credentials locally. While the granular permissions for personal access tokens don’t add a lot of benefit in this case, tokens do feel a little bit easier to manage than SSH keys.