bundlerEnv: can’t use more than one gem from the same Git repo

[amiryal]

Describe the bug

As was already noted before in #59284, it is not possible currently to build a bundlerEnv derivation from a Gemfile.lock that depends on more than one gem from the same Git repository.

Steps To Reproduce

1. Create a Gemfile:

source 'https://rubygems.org/'

gem 'activemodel', github: 'rails/rails' # depends on 'activesupport' from the same repo

2. Generate gemset.nix:

$ BUNDLE_FORCE_RUBY_PLATFORM=1 nix run nixpkgs#bundler lock
$ nix run nixpkgs#bundix

3. Attempt to use the gemset:

$ nix-shell -I nixpkgs="$(nix flake metadata --json nixpkgs | nix run nixpkgs#jq -- .path -r)" -p 'bundlerEnv { name = "test"; gemdir = ./.; }'

This fails with the error message:

error: collision between `/nix/store/z8gyqpp6ci9fwy9snm2rjr7wfmjrirbv-ruby3.1-activesupport-f698d735bfd2/lib/ruby/gems/3.1.0/bundler/gems/rails-f698d735bfd2/.git/index' and `/nix/store/w0nvflbz30paiad7ksskggbzkrb0sgg1-ruby3.1-activemodel-f698d735bfd2/lib/ruby/gems/3.1.0/bundler/gems/rails-f698d735bfd2/.git/index'

I.e. both gems contain the same /lib/ruby/gems/3.1.0/bundler/gems/rails-f698d735bfd2/.git/ directory, causing a collision.

Expected behavior

Most importantly, it should not fail, no matter the implementation detail of the solution.

A secondary goal may be to save some space by deduplication, as suggested by https://github.com/NixOS/nixpkgs/pull/59284#issuecomment-481916902.

Notify maintainers

@manveru @zimbatm

Metadata

$ nix-shell -I nixpkgs="$(nix flake metadata --json nixpkgs | nix run nixpkgs#jq -- .path -r)" -p nix-info --run 'nix-info -m'
 - system: `"x86_64-linux"`
 - host os: `Linux 6.8.0-41-generic, Ubuntu, 24.04.1 LTS (Noble Numbat), nobuild`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - nixpkgs: `not found`

$ nix flake metadata --json nixpkgs | nix run nixpkgs#jq -- .resolvedUrl -r
github:NixOS/nixpkgs/13fe00cb6c75461901f072ae62b5805baef9f8b2

[amiryal]

Hi @burke. Would you mind sharing, if you managed to find a solution or a nice workaround?

[burke]

I remember struggling with this and ripping apart a bunch of code to try to make this not redundantly clone the same repo... if I ended up coming to a solution there, it was not clean.

IMO the nix/ruby stuff all pretty much needs a rewrite.

[amiryal]

I’ve found a workaround.

Instead of this:

bundlerEnv {
  attrs …;
}

Do this:

bundlerEnv {
  attrs …;
  ignoreCollisions = true;
}

Now I am wondering if the default in nixpkgs should be changed from ignoreCollisions ? false to ignoreCollisions ? true, since Bundler itself doesn’t seem to care anyway (e.g., when more than one gem in the bundle provides the same bin/ script, one gem wins and the rest are ignored). If not that, then at least the message from buildEnv should be changed, to something that alludes to the ignoreCollisions argument, as a helpful hint.

[amiryal]

Although Bundler itself (not when using nixpkgs bundlerEnv) does issue a warning when running that conflicting script:

$ bundle exec console
The `console` executable in the `taskinator` gem is being loaded, but it's also present in other gems (plagiarism-checker, rocketchat).
If you meant to run the executable for another gem, make sure you use a project specific binstub (`bundle binstub <gem_name>`).
If you plan to use multiple conflicting executables, generate binstubs for them and disambiguate their names.
[1] pry(main)>