uninitialized constant Bundler::RubygemsIntegration::Gem

[benoittgt]

Hello

With the last version of Bundler we can see this error on RSpec-rails CI.

rm config/application.rb
** Invoke generate:stuff (first_time)
** Execute generate:stuff
bin/rake app:template LOCATION='../../example_app_generator/generate_stuff.rb'
    generate  rspec:install
Traceback (most recent call last):
    7: from bin/rails:1:in `require'
    6: from /Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.0/lib/bundler/setup.rb:3:in `<top (required)>'
    5: from /Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.0/lib/bundler/setup.rb:3:in `require_relative'
    4: from /Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.0/lib/bundler/shared_helpers.rb:8:in `<top (required)>'
    3: from /Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.0/lib/bundler/shared_helpers.rb:8:in `require_relative'
    2: from /Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.0/lib/bundler/rubygems_integration.rb:3:in `<top (required)>'
    1: from /Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.0/lib/bundler/rubygems_integration.rb:4:in `<module:Bundler>'
/Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.0/lib/bundler/rubygems_integration.rb:616:in `<class:RubygemsIntegration>': uninitialized constant Bundler::RubygemsIntegration::Gem (NameError)
rake aborted!
Command failed with status (1): [bin/rake app:template LOCATION='../../exam...]
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/file_utils.rb:67:in `block in create_shell_runner'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/file_utils.rb:57:in `sh'
/Users/bti/code/rspec-dev/repos/rspec-rails/Rakefile:109:in `block (2 levels) in in_example_app'
/Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler.rb:309:in `block in with_original_env'
/Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler.rb:562:in `with_env'
/Users/bti/.rvm/gems/ruby-2.6.3/gems/bundler-2.0.2/lib/bundler.rb:309:in `with_original_env'
/Users/bti/code/rspec-dev/repos/rspec-rails/Rakefile:107:in `block in in_example_app'
/Users/bti/code/rspec-dev/repos/rspec-rails/Rakefile:106:in `chdir'
/Users/bti/code/rspec-dev/repos/rspec-rails/Rakefile:106:in `in_example_app'
/Users/bti/code/rspec-dev/repos/rspec-rails/Rakefile:96:in `block (2 levels) in <top (required)>'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `block in execute'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `each'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:273:in `execute'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:214:in `block in invoke_with_call_chain'
/Users/bti/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:238:in `block in invoke_prerequisites'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:236:in `each'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:236:in `invoke_prerequisites'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:213:in `block in invoke_with_call_chain'
/Users/bti/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:238:in `block in invoke_prerequisites'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:236:in `each'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:236:in `invoke_prerequisites'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:213:in `block in invoke_with_call_chain'
/Users/bti/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/task.rb:183:in `invoke'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:160:in `invoke_task'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:116:in `each'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:116:in `block in top_level'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:125:in `run_with_threads'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:110:in `top_level'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:83:in `block in run'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:186:in `standard_exception_handling'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/lib/rake/application.rb:80:in `run'
/Users/bti/code/rspec-dev/repos/bundle/ruby/2.6.0/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
bin/rake:29:in `load'
bin/rake:29:in `<main>'

I don't have the issue if I jump to a previous Bundler version. I will keep digging and will update this ticket with relevant informations.

I don't have the issue if I explicitly require 'rubygems' before calling https://github.com/bundler/bundler/blob/bada03dd6d4d15828fb5b2cf7f744948e88a69a3/lib/bundler/rubygems_integration.rb#L587

It seems their is a loading issue somewhere.

Thanks

[deivid-rodriguez]

A first guess after a quick look:

This would happen if, while using the --disable-gems ruby flag, you do require "bundler"; Bundler.with_original_env {}.

It sounds like we're missing a require "rubygems" on top of this file. I'm pretty sure it existed but we removed it because we thought "why using bundler if you don't want to use gems?".

I think maybe you can fix this with something like

 --git a/script/custom_build_functions.sh b/script/custom_build_functions.sh
index f8b0f06b..d550fa12 100644
--- a/script/custom_build_functions.sh
+++ b/script/custom_build_functions.sh
@@ -1,6 +1,6 @@
 function run_cukes {
   if is_mri_192_plus; then
-    bin/rake acceptance --trace
+    eval "(unset RUBYOPT; exec bin/rake acceptance --trace)"
     return $?
   elif is_jruby; then
     bin/rake smoke:app

But we should probably still restore the require "rubygems" to save other people from this trouble.

[benoittgt]

The command fail when executed by rake:

      sh "bundle exec rails new ./tmp/example_app --no-rc --skip-javascript --skip-bootsnap -skip-sprockets --skip-git --skip-test-unit --skip-listen --skip-bundle --template=example_app_generator/generate_app.rb"

But alone in the shell. I have no issues.

---

Ok your comment make sense. Bundle alone does not use export RUBYOPT="--disable=gem".

Feel free to close this ticket.

[deivid-rodriguez]

@benoittgt Does it work if you lazily load things like this?

diff --git a/lib/bundler/rubygems_integration.rb b/lib/bundler/rubygems_integration.rb
index 3ca1831dee..213562db8d 100644
--- a/lib/bundler/rubygems_integration.rb
+++ b/lib/bundler/rubygems_integration.rb
@@ -584,12 +584,10 @@ module Bundler
       Gem::Specification.stubs_for(name).map(&:to_spec)
     end

-    if Gem::Specification.respond_to?(:default_stubs)
-      def default_stubs
+    def default_stubs
+      if Gem::Specification.respond_to?(:default_stubs)
         Gem::Specification.default_stubs("*.gemspec")
-      end
-    else
-      def default_stubs
+      else
         Gem::Specification.send(:default_stubs, "*.gemspec")
       end
     end

[JonRowe]

The backstory of why we (rspec) do this, we've historically disabled rubygems for two reasons, bundlers binstubs are faster, and because its possible for some people to do it and we'd like to make sure rspec runs without a requirement on rubygems/bundler.

Do we need to modify our approach now?

[deivid-rodriguez]

No, I'm fine with changing things in bundler to fit your needs again, it's just a require "rubygems" after all.

I'm curious though whether the previous patch I posted fixes things for you, or just moves the error to somewhere else. Since you seem to only be using Bundler.with_original_env, and that only changes the environment, it might not require rubygems at all.

[benoittgt]

With unset RUBYOPT it's working.

[deivid-rodriguez]

I meant this patch.

[deivid-rodriguez]

Will be fixed by #7505.

[JonRowe]

After we unset rubyopts, we have an issue in our build with aruba, which runs commands in sub rails projects (yo dawg etc) in order to verify our rspec-rails behaviour, we now get:

You have already activated bundler 2.1.0, but your Gemfile requires bundler 2.1.1.
Since bundler is a default gem, you can either remove your dependency on it or
try updating to a newer version of bundler that supports bundler as a default gem.

I can open a new issue for this, but I would kind of expect bundler to support itself as a default gem :joy:

[deivid-rodriguez]

Yeah, the last two lines of the message don't make much sense to me either :laughing:. Can you open a new issue with repro steps, I guess this can be currently reproduced in your rspec-rails CI, right?

[JonRowe]

You can see it here, https://travis-ci.org/rspec/rspec-rails/jobs/626903497, its late here but I'll see if I can extract a repro tomorrow, its likely its an interaction with aruba not providing us a clean environment, are there new bundler environment variables in 2.x?

[deivid-rodriguez]

Thanks, I'll try to have a look tomorrow, the CI config should be enough but of course if it can be minimized I appreciate it.

are there new bundler environment variables in 2.x?

No, I don't think so. But there are many changes, master had been unreleased for a very long time... :(

[benoittgt]

I meant this patch.

I just tried and it seems that the patch works.

[deivid-rodriguez]

Interesting, with the test I'm adding for #7505, it leads to a different error so I went with adding the require.

[benoittgt]

The setup with the full RSpec build script with a local version of bundler is not easy. I probably messed up somewhere if you have different result. Sorry.

[deivid-rodriguez]

I see, no problem, thanks for trying anyways.

[hsbt]

@benoittgt I totally wonder why the test suite of rspec-rails invoked with --disable-gems flag. Rails always used bundler in their internal. So, Rails always needs to RubyGems. There is no reason to add --disable-gems flag in test suite of rspec-rails

[JonRowe]

So, Rails always needs to RubyGems. There is no reason to add --disable-gems flag in test suite of rspec-rails

Rails always needs it, but we don't, and invoking bundler bin stubs is faster than bundle exec.

[benoittgt]

Thanks @hsbt for your comment. @JonRowe gave you the answer to your question. 😊

Also the --disable-gems is written here https://github.com/rspec/rspec-dev/blob/master/travis/script/functions.sh#L14. We share those scripts accross all RSpec gems (rspec-core, rspec-mocks, etc).

[hsbt]

@JonRowe @benoittgt I think It's the configuration issue of rspec. Bundler 2.1.2 always show circular require warnings with RUBYGEMS_GEMDEPS environmental variable.

https://github.com/ruby/ruby/commit/75dca097b873222eaf3c70b7f43e3ffb98b7f2ed

I understood rspec need to run without rubygems, But I'm not sure why adding --disable-gems flag for the rspec-rails test suite.

[deivid-rodriguez]

Yes, I agree this is a configuration problem with rspec-rails CI (I think they fixed it, anyways?). But I didn't want to break other similar setups that might be doing similar things, specially since it was just a require.

But I think we could start warning people about using bundler in combination with --disable-gems, and drop this in future versions.

[JonRowe]

But I'm not sure why adding --disable-gems flag for the rspec-rails test suite.

To find problems like this, as it is possible to disable rubygems, we do so. We also benchmarked it to be faster. If bundler needs rubygems for something it should require it. It's not up to us to do so, as its possible for a user not to have done so.

But I think we could start warning people about using bundler in combination with --disable-gems, and drop this in future versions.

Why? If a simple require solves the problem have the require... Its the least surprising option and the least work for your users. Let Ruby decide if rubygems is optional.

I'm biased having been exhausted by all the different workarounds we do for ruby / rubygems / bundler combinations but bundler is a core tenement of the Ruby ecosystem, please consider that like us you need to cater for the lowest common denominator and not put up more barriers for people to work around.

[deivid-rodriguez]

There's bigger fish to fry, so without prolonging the discussion of whether this is a issue with rspec-rails configuration or not, I agree with keeping the require and workaround for the circular require issue without deprecating anything.

[hsbt]

FYI: --disable-gems is NOT a performance improvement option. It only affects less 0.1 sec. So --disable-gems is debugging option for ruby-core development like gdb or etc.

[JonRowe]

FYI: --disable-gems is NOT a performance improvement option. It only affects less 0.1 sec. So --disable-gems is debugging option for ruby-core development like gdb or etc.

FWIW the main reason we do it is to take the lowest common denominator approach. It is possible to do so and we have historically had bug reports from it (I believe one of the linux's didn't have it installed by default at one point?) so we test without it. We also avoid requiring the standard library unless absolutely necessary for the same reason, it is possible for it not to be there and for it to cause false positives.

This philosophy, of supporting the maximum amount of the Ruby community is why we still support 1.8.7, even though that will be (finally) going away in RSpec 4.

[hsbt]

Again. I didn't say about rspec. I didn't understand to use --disable-gems option with rspec-rails.

Rails use rubygems and bundler explicitly. But rspec-rails uses --disable-gems option for avoiding rubygems and relying to load rubygems on bundler. I still wonder why rspec-rails didn't load rubygems in themselves.

[JonRowe]

Rails use rubygems and bundler explicitly. But rspec-rails uses --disable-gems option for avoiding rubygems and relying to load rubygems on bundler. I still wonder why rspec-rails didn't load rubygems in themselves.

Because rspec-rails follows the same philosophy as rspec.

As we don't ourselves use rubygems, we don't require it. It is up to bundler to require rubygems if it needs it.

[hsbt]

okay. I understood the opinion of the rspec team. But I totally against it.

I'm +1 to https://github.com/bundler/bundler/issues/7487#issuecomment-569905840 for this issue.