rubygems / bundler

Manage your Ruby application's gem dependencies
https://bundler.io
MIT License
4.88k stars 1.99k forks source link

`cannot load such file -- did_you_mean` on `bunde install` with ruby-2.7.0 #7642

Closed ojab closed 4 years ago

ojab commented 4 years ago

I have strange issue with bundle install: we have centos-7 docker image with ruby built from fedora specs and it works perfectly fine with ruby-2.6.5 & bundler-2.1.2. With ruby-2.7.0 installation of any gem with native extension fails with

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /tmp/testapp/.bundle/ruby/2.7.0/gems/pg-1.2.2/ext
/usr/bin/ruby -I /usr/share/rubygems -r ./siteconf20200217-143-b60dhy.rb extconf.rb --with-pg-config\=/usr/pgsql-11/bin/pg_config
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- did_you_mean (LoadError)
    from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:92:in `require'
    from <internal:gem_prelude>:2:in `<internal:gem_prelude>'

extconf failed, exit code 1

Gem files will remain installed in /tmp/testapp/.bundle/ruby/2.7.0/gems/pg-1.2.2 for inspection.
Results logged to /tmp/testapp/.bundle/ruby/2.7.0/extensions/x86_64-linux/2.7.0/pg-1.2.2/gem_make.out
`--verbose` output: ``` $ bundle --path=.bundle --verbose Running `bundle install --path ".bundle" --verbose` with bundler 2.1.4 Found changes from the lockfile, re-resolving dependencies because the list of sources changed, the dependencies in your gemfile changed, you added a new platform to your gemfile HTTP GET https://index.rubygems.org/versions HTTP 206 Partial Content https://index.rubygems.org/versions Fetching gem metadata from https://rubygems.org/ Looking up gems ["pg"] Looking up gems ["rake-compiler"] Looking up gems ["rake"] Resolving dependencies... Using bundler 2.1.4 0: bundler (2.1.4) from /usr/local/share/gems/specifications/bundler-2.1.4.gemspec Fetching pg 1.2.2 Installing pg 1.2.2 with native extensions Building native extensions with: '--with-pg-config=/usr/pgsql-11/bin/pg_config' This could take a while... Gem::Ext::BuildError: ERROR: Failed to build gem native extension. current directory: /tmp/testapp/.bundle/ruby/2.7.0/gems/pg-1.2.2/ext /usr/bin/ruby -I /usr/share/rubygems -r ./siteconf20200217-201-1r8tqhb.rb extconf.rb --with-pg-config\=/usr/pgsql-11/bin/pg_config /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- did_you_mean (LoadError) from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:92:in `require' from :2:in `' extconf failed, exit code 1 Gem files will remain installed in /tmp/testapp/.bundle/ruby/2.7.0/gems/pg-1.2.2 for inspection. Results logged to /tmp/testapp/.bundle/ruby/2.7.0/extensions/x86_64-linux/2.7.0/pg-1.2.2/gem_make.out /usr/share/rubygems/rubygems/ext/builder.rb:97:in `run' /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:48:in `block in build' /usr/share/ruby/tempfile.rb:291:in `open' /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:30:in `build' /usr/share/rubygems/rubygems/ext/builder.rb:167:in `block in build_extension' /usr/share/rubygems/rubygems/ext/builder.rb:163:in `synchronize' /usr/share/rubygems/rubygems/ext/builder.rb:163:in `build_extension' /usr/share/rubygems/rubygems/ext/builder.rb:208:in `block in build_extensions' /usr/share/rubygems/rubygems/ext/builder.rb:205:in `each' /usr/share/rubygems/rubygems/ext/builder.rb:205:in `build_extensions' /usr/share/rubygems/rubygems/installer.rb:844:in `build_extensions' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/rubygems_gem_installer.rb:23:in `build_extensions' /usr/share/rubygems/rubygems/installer.rb:327:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:160:in `block in install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:228:in `preserve_paths' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:149:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/gem_installer.rb:67:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/gem_installer.rb:72:in `block in install_with_settings' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:589:in `install_with_build_args' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/gem_installer.rb:72:in `install_with_settings' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/gem_installer.rb:18:in `install_from_spec' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:160:in `do_install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:144:in `install_serially' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:99:in `call' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:78:in `call' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:282:in `install_in_parallel' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:197:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:92:in `block in run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/process_lock.rb:12:in `block in lock' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/process_lock.rb:9:in `open' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/process_lock.rb:9:in `lock' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:73:in `run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:25:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli/install.rb:66:in `run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:256:in `block in install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/settings.rb:124:in `temporary' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:255:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:30:in `dispatch' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:24:in `start' /usr/local/share/gems/gems/bundler-2.1.4/exe/bundle:46:in `block in ' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors' /usr/local/share/gems/gems/bundler-2.1.4/exe/bundle:34:in `' /usr/local/bin/bundle:23:in `load' /usr/local/bin/bundle:23:in `
' An error occurred while installing pg (1.2.2), and Bundler cannot continue. Make sure that `gem install pg -v '1.2.2' --source 'https://rubygems.org/'` succeeds before bundling. In Gemfile: pg Bundler::InstallError: Gem::Ext::BuildError: ERROR: Failed to build gem native extension. current directory: /tmp/testapp/.bundle/ruby/2.7.0/gems/pg-1.2.2/ext /usr/bin/ruby -I /usr/share/rubygems -r ./siteconf20200217-201-1r8tqhb.rb extconf.rb --with-pg-config\=/usr/pgsql-11/bin/pg_config /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- did_you_mean (LoadError) from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:92:in `require' from :2:in `' extconf failed, exit code 1 Gem files will remain installed in /tmp/testapp/.bundle/ruby/2.7.0/gems/pg-1.2.2 for inspection. Results logged to /tmp/testapp/.bundle/ruby/2.7.0/extensions/x86_64-linux/2.7.0/pg-1.2.2/gem_make.out /usr/share/rubygems/rubygems/ext/builder.rb:97:in `run' /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:48:in `block in build' /usr/share/ruby/tempfile.rb:291:in `open' /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:30:in `build' /usr/share/rubygems/rubygems/ext/builder.rb:167:in `block in build_extension' /usr/share/rubygems/rubygems/ext/builder.rb:163:in `synchronize' /usr/share/rubygems/rubygems/ext/builder.rb:163:in `build_extension' /usr/share/rubygems/rubygems/ext/builder.rb:208:in `block in build_extensions' /usr/share/rubygems/rubygems/ext/builder.rb:205:in `each' /usr/share/rubygems/rubygems/ext/builder.rb:205:in `build_extensions' /usr/share/rubygems/rubygems/installer.rb:844:in `build_extensions' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/rubygems_gem_installer.rb:23:in `build_extensions' /usr/share/rubygems/rubygems/installer.rb:327:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:160:in `block in install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:228:in `preserve_paths' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/source/rubygems.rb:149:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/gem_installer.rb:67:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/gem_installer.rb:72:in `block in install_with_settings' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:589:in `install_with_build_args' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/gem_installer.rb:72:in `install_with_settings' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/gem_installer.rb:18:in `install_from_spec' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:160:in `do_install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:144:in `install_serially' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:99:in `call' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:78:in `call' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:282:in `install_in_parallel' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:197:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:92:in `block in run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/process_lock.rb:12:in `block in lock' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/process_lock.rb:9:in `open' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/process_lock.rb:9:in `lock' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:73:in `run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:25:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli/install.rb:66:in `run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:256:in `block in install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/settings.rb:124:in `temporary' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:255:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:30:in `dispatch' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:24:in `start' /usr/local/share/gems/gems/bundler-2.1.4/exe/bundle:46:in `block in ' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors' /usr/local/share/gems/gems/bundler-2.1.4/exe/bundle:34:in `' /usr/local/bin/bundle:23:in `load' /usr/local/bin/bundle:23:in `
' An error occurred while installing pg (1.2.2), and Bundler cannot continue. Make sure that `gem install pg -v '1.2.2' --source 'https://rubygems.org/'` succeeds before bundling. In Gemfile: pg /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:197:in `handle_error' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:102:in `call' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer/parallel_installer.rb:78:in `call' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:282:in `install_in_parallel' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:197:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:92:in `block in run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/process_lock.rb:12:in `block in lock' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/process_lock.rb:9:in `open' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/process_lock.rb:9:in `lock' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:73:in `run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/installer.rb:25:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli/install.rb:66:in `run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:256:in `block in install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/settings.rb:124:in `temporary' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:255:in `install' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:30:in `dispatch' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:24:in `start' /usr/local/share/gems/gems/bundler-2.1.4/exe/bundle:46:in `block in ' /usr/local/share/gems/gems/bundler-2.1.4/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors' /usr/local/share/gems/gems/bundler-2.1.4/exe/bundle:34:in `' /usr/local/bin/bundle:23:in `load' /usr/local/bin/bundle:23:in `
' ```

even though did_you_mean gem is installed and

[containeruser@b05ece9058ff testapp]$ ruby -e "require 'did_you_mean'"
[containeruser@b05ece9058ff testapp]$ echo $?
0

If I set debugger @ /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:92did_you_mean is actually required twice and first time it's required just ok, only second time fails. BUNDLER envvars are empty first time, second time it is

"BUNDLER_ORIG_BUNDLE_BIN_PATH"=>"BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL",
"BUNDLER_ORIG_BUNDLE_GEMFILE"=>"BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL",
"BUNDLER_ORIG_BUNDLER_VERSION"=>"BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL",
"BUNDLER_ORIG_GEM_HOME"=>"BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL",
"BUNDLER_ORIG_GEM_PATH"=>"BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL",
"BUNDLER_ORIG_MANPATH"=>"BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL",
"BUNDLER_ORIG_PATH"=>"/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/containeruser/.local/bin:/home/containeruser/bin",
"BUNDLER_ORIG_RB_USER_INSTALL"=>"BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL",
"BUNDLER_ORIG_RUBYLIB"=>"BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL",
"BUNDLER_ORIG_RUBYOPT"=>"BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL"

Any hints how to fix/debug it are welcome.

deivid-rodriguez commented 4 years ago

Can you also check RUBYOPT and RUBYLIB?

ojab commented 4 years ago

Empty on both runs.

deivid-rodriguez commented 4 years ago

Coulld you run ruby -e "require 'did_you_mean'; puts $LOADED_FEATURES.grep(/did_you_mean\.rb/)"?

ojab commented 4 years ago
Traceback (most recent call last):
-e:1:in `<main>': undefined method `grep' for nil:NilClass (NoMethodError)
ojab commented 4 years ago

If I run it from irb:

irb(main):001:0> require 'did_you_mean'; puts $LOADED_FEATURES.grep(/did_you_mean\.rb/)
/usr/share/gems/gems/did_you_mean-1.4.0/lib/did_you_mean.rb
=> nil
deivid-rodriguez commented 4 years ago

So if you run $ gem list | grep default, did_you_mean is not there, right?

It sounds like your ruby distribution doesn't ship did_you_mean as a default gem, nor as a default library, and ruby pretty much expects this.

How did you install ruby?

As a workaround, I guess you could try RUBYOPT=--disable-did_you_mean.

ojab commented 4 years ago

Basically ruby is built from this spec. So after a bit of googling I've found https://github.com/ruby/did_you_mean/issues/117#issuecomment-482733159 and looks like https://github.com/ruby/ruby/commit/171803d5d34feb1b4244ca81b9db0a7bc2171c85 should've made it default gem, but somehow it's not in fedora's 2.7.0. Thanks for the help, I'll open a bug in fedora bugtracker.

deivid-rodriguez commented 4 years ago

Great, thanks.

I guess we could avoid loading did_you_mean here like:

diff --git a/lib/rubygems/ext/ext_conf_builder.rb b/lib/rubygems/ext/ext_conf_builder.rb
index 88be7ecfe8..0f9d40210a 100644
--- a/lib/rubygems/ext/ext_conf_builder.rb
+++ b/lib/rubygems/ext/ext_conf_builder.rb
@@ -40,7 +40,7 @@ class Gem::Ext::ExtConfBuilder < Gem::Ext::Builder
       destdir = ENV["DESTDIR"]

       begin
-        cmd = Gem.ruby.shellsplit << "-I" << File.expand_path("../../..", __FILE__) <<
+        cmd = Gem.ruby.shellsplit << "--disable-did_you_mean" << "-I" << File.expand_path("../../..", __FILE__) <<
               "-r" << get_relative_path(siteconf.path) << File.basename(extension)
         cmd.push(*args)

But still Fedora should probably change this.

deivid-rodriguez commented 4 years ago

I'm pretty sure they are already aware though, since they proposed https://github.com/ruby/ruby/pull/2764 to ruby-core.

ojab commented 4 years ago

Oh, good. Then I workaround it in our images for the time being.