Bundler should give a better error when the Gemfile itself activates a conflicting gem

[Bandes]

I have been running into a strange situation which I believe is due to differing versions of the built-in gem set in different docker images.

My application runs on ruby 3.0.6 currently. My CI is CircleCI, and I use the cimg/ruby:3.0.6-browsers which appears to use set:1.0.3 However my devops team is running my application on Docker, using ruby:3.0.6alpine3.16 which appears to use set:1.0.1

Whichever of those versions I specify in Gemfile.lock will cause an error similar to this:

`check_for_activated_spec!': You have already activated set 1.0.1, but your Gemfile requires set 1.0.3. Since set is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports set as a default gem. (Gem::LoadError)

Similar things have happened to me on other projects that use more up-to-date versions of ruby.

I would have expected that bundler would be able to force the use of the version of the gem in Gemfile.lock, it hadn't occurred to me that I would need to inspect which versions were built-in to my images.

[deivid-rodriguez]

We should be supporting gemfied set since Bundler 2.2.8, which first included https://github.com/rubygems/rubygems/pull/4297. Would it be possible that you are using an older Bundler?

[Bandes]

I don't think so. My Gemfile.lock was built with 2.4.22

RUBY VERSION
   ruby 3.0.6p216

BUNDLED WITH
   2.4.22

[deivid-rodriguez]

Not a good guess then, thanks for confirming.

Please provide repro steps so that we can take a look at this.

[Bandes]

I'm having trouble finding ways to describe how to reproduce the issue without linking to our specific code or our specific images. I do have this which shows the docker versions we are using - the base image has 2.2.33 which we are updating to 2.4.22 before we bundle install

╰─ docker run --rm -it public.ecr.aws/docker/library/ruby:3.0.6-alpine3.16 sh                                                                                                                                                                                                                                                                                          ─╯
/ # bundle --version
Bundler version 2.2.33
/ # gem install bundler:2.4.22
Fetching bundler-2.4.22.gem
Successfully installed bundler-2.4.22
1 gem installed
/ # bundle --version
Bundler version 2.4.22
/ # gem list | grep set
set (default: 1.0.1)

[deivid-rodriguez]

Maybe extracting only dependency manifests to a separate repository, removing your private code and private gems, and see if it still reproduces?

[indirect]

I have suddenly started seeing this problem on a new machine, but I'm having trouble creating another copy of the repo with the same problem. The symptom I can observe is this error during Bundler.setup:

❯ b ruby -e "puts 'hi'"
/Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:304:in `check_for_activated_spec!': You have already activated date 3.3.4, but your Gemfile requires date 3.3.3. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:25:in `block in setup'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `map'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `setup'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler.rb:162:in `setup'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `block in <top (required)>'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:159:in `with_level'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:111:in `silence'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `<top (required)>'
        from <internal:/Users/andre/.rubies/ruby-3.2.2/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from <internal:/Users/andre/.rubies/ruby-3.2.2/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from /Users/andre/.rubies/ruby-3.2.2/lib/ruby/3.2.0/rubygems.rb:1370:in `<top (required)>'
        from <internal:gem_prelude>:2:in `require'
        from <internal:gem_prelude>:2:in `<internal:gem_prelude>'

The error only appears when installing to system gems, so I can make it appear and disappear repeatedly by running bundle config path .bundle (gone) and then bundle config unset path (error returns).

My system gems include both the locked version of date and a newer version:

❯ gem list date

*** LOCAL GEMS ***

date (3.3.4, default: 3.3.3)

[indirect]

oh no 🤦🏻 I figured it out. my Rails app contains a (relatively) complicated gemfile that loads some ruby to evaluate whether to include other gems. The "configuration" class has this at the top:

require "erb"
require "open-uri"
require "psych"
require "set"

Turns out open-uri requires time, which requires date. So ultimately what this boils down to is exactly what the error message says: somehow, you have a newer version of this gem installed, and merely evaluating your Gemfile has required a version that does not match your lock. So Bundler explodes when it gets around to trying to load your locked version of the gem (whether that gem is set or date).

We should probably turn this into one of the advanced Bundler error messages, and say something like this:

Your Gemfile is locked to date version 3.3.3. When Bundler tried to require date, it discovered that other Ruby code had already required date version 3.3.4. As a result, it is impossible to set up your Gemfile.

This is most likely caused by your Gemfile running Ruby code that requires gems, which is not supported. You will need to remove whatever is inside your Gemfile requiring gems, so that Bundler can take care of loading the versions you need.

Maybe this is also an opportunity for us to overwrite require to raise an exception while we eval Gemfiles? At least then it would fail early, and let users know exactly where the bad code is. 😅 cc @deivid-rodriguez @martinemde

[indirect]

The good news is that figuring all of this out suggests a workaround:

1. bundle config path .bundle to create a separate space for gems just for the given app.
2. bundle install to install gems just for this app, without any newer gems available to be required while evaluating the gemfile.
3. bundle clean after every bundle update, to make sure that newer/older gems are not also installed into the separate app gem space to be picked up by the require lines in the gemfile.

I have tested this on my example app, and it seems to work.

[deivid-rodriguez]

The idea of giving a better error message when a require fails with this error during Gemfile evaluation makes sense to me 👍.

[martinemde]

I made a branch with a failing test for this case. I'm not confident about its approach, but it does trigger the error: https://github.com/rubygems/rubygems/compare/require-conflict-in-gemfile

[Bandes]

Hi this repo https://github.com/modolabs/bundler-set-error-example should demonstrate the issue I am having. It seems like it might have something to do with spring because if I remove those gems from the gemfile I can no longer reproduce the issue. It might also be a problem with the dockerfile, but I'm not sure.

If these commands are executed in that repo, the error will occur for me

docker build --no-cache --build-arg BUNDLE_WITHOUT=nil . -t test-project
docker run -t test-project bundle exec rails runner '1+1'

[deivid-rodriguez]

In your case, you're using a very old spring version that does not include https://github.com/rails/spring/pull/659, that's why you run into this error. If you upgrade (or remove) spring, you should be fine!

[stanhu]

This might be a separate issue, but I ran into a similar issue with the uri gem that is loaded by RubyGems only when:

1. A local gemspec is used.
2. The gemspec has a home page URL: https://github.com/ruby/ruby/blob/10b9679fa61f40c08ec28ec34251f7ec8648054d/lib/rubygems/specification_policy.rb#L430.

Consider this Gemfile:

source 'https://rubygems.org'

gem 'uri', '= 0.12.2'
gem 'test', path: 'test'

In test/test.gemspec:

# frozen_string_literal: true

Gem::Specification.new do |s|
  s.name = "test"
  s.version = "1.0.0"
  s.summary = "test"
  s.authors = ['John Doe']
  s.homepage = 'https://example.com'
end

If I simply run bundle exec irb, I see:

/Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:304:in `check_for_activated_spec!': You have already activated uri 0.13.0, but your Gemfile requires uri 0.12.2. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:25:in `block in setup'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `map'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `setup'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler.rb:162:in `setup'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `block in <top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:159:in `with_level'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:111:in `silence'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `<top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:56:in `require_relative'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:56:in `kernel_load'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:23:in `run'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:492:in `exec'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:34:in `dispatch'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:28:in `start'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/exe/bundle:37:in `block in <top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/exe/bundle:29:in `<top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/bin/bundle:25:in `load'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/bin/bundle:25:in `<main>'

In this example, the user doesn't have a whole lot of control over the loading of uri.

The only workaround I can see is to update uri to v0.13.0.

[deivid-rodriguez]

Yeah, we should vendor the uri gem in Rubygems to avoid this problem.

[deivid-rodriguez]

Closed unintentionally by #7386. Reopening.

[N0xFF]

After updating to Rails 7.1 we started receiving an error:

/Ruby/3.1.4/x64/lib/ruby/gems/3.1.0/gems/bundler-2.5.6/lib/bundler/runtime.rb:304:in `check_for_activated_spec!': You have already activated mutex_m 0.1.1, but your Gemfile requires mutex_m 0.2.0. Since mutex_m is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports mutex_m as a default gem. (Gem::LoadError)

The cause was Spring. After disabling it the error is gone, but investigating the reason was hard. It would be better if we had a clearer error message.

[joshuapinter]

FYI, we were receiving the following error:

Error: The application encountered the following error: You have already activated strscan 3.0.7, but your Gemfile requires strscan 3.1.0. Since strscan is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports strscan as a default gem. (Gem::LoadError)

What worked was manually install strscan at 3.1.0 in the global gem space:

sudo gem install strscan:3.1.0

This ensures that one is used instead of the default 3.0.7:

gem list strscan

*** LOCAL GEMS ***

strscan (3.1.0, default: 3.0.7)

I tried other methods before doing this that I saw elsewhere, including:

1. Upgrading rubygems.
2. Setting this NGINX config: passenger_env_var RUBYOPT '-r bundler/setup';

Neither worked.

I did get it working by pinning my version of strscan to 3.0.7 in our Gemfile but that wasn't a long term solution because we needed to upgrade rexml due to a security risk and 3.0.7 depended on an older version of rexml, so we needed to upgrade strscan.

Hope that helps others.

[pboling]

I have this issue in GitHub Actions CI on a public gem, with, and without, a lockfile.

In both cases the error looks like this:

bundler: failed to load command: rake (/home/runner/work/rots/rots/vendor/bundle/ruby/2.7.0/bin/rake)
/opt/hostedtoolcache/Ruby/2.[7](https://github.com/oauth-xx/rots/actions/runs/11023286935/job/30614306104#step:4:8).8/x64/lib/ruby/gems/2.7.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:304:in `check_for_activated_spec!': You have already activated uri 0.10.0.2, but your Gemfile requires uri 0.13.1. Since uri is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports uri as a default gem. (Gem::LoadError)

Scenario	commit	build
No Lockfile	commit	build failure
With Lockfile	commit	build failure

you can either remove your dependency on it or try updating to a newer version of bundler that supports uri as a default gem

Of these options, neither are great, because uri isn't even a direct dependency (it it pulled in by net-http, and I am on the latest version of bundler that will work on Ruby 2.7.8.

I'll try removing net-http, but that will mean that it won't be simple to make a single Ruby library compatible with both Ruby 2.7 and Ruby 3.5, when the Ruby former-stdlib gem deprecations end, and become real errors.

[pboling]

I think this means that if any Ruby library wants to easily support Ruby 2.7 - Ruby 3.5, of which I expect there will be many, to ease the transition, we might need a new version of bundler 2.4 which fixes this.

[deivid-rodriguez]

Usually @hsbt handles backports, but since Ruby 2.7 has already reached its end of life, I would personally not backport this fix and let gem maintainers drop support for Ruby 2.7 instead.

[hsbt]

Right. If you pick the related commits or PRs, I will backport to bundlr 2.4 or 2.3 and merge them into Ruby 3.3 and 3.2.