ruby - gems are incompatible with other patch-level versions of ruby

[jdelStrother]

Describe the bug

Nix's ruby seems quite brittle - if I install a gem with C extensions and then switch to a different nix ruby version (even with just a patch-level version difference), it fails to load the gem with incompatible library version. This can make gem entirely unusable if, say, the stringio gem is installed in GEM_PATH:

$ gem list
<internal:/nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:88:in `require': incompatible library version - /root/nixpkgs/bundle/gems/stringio-3.0.5/lib/stringio.so (LoadError)
    from <internal:/nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:88:in `require'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych/nodes/node.rb:2:in `<top (required)>'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych/nodes.rb:2:in `require_relative'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych/nodes.rb:2:in `<top (required)>'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych.rb:15:in `require_relative'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych.rb:15:in `<top (required)>'
    from <internal:/nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:151:in `require'
    from <internal:/nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:151:in `require'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems.rb:608:in `load_yaml'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/config_file.rb:346:in `load_file'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/config_file.rb:189:in `initialize'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/gem_runner.rb:71:in `new'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/gem_runner.rb:71:in `do_configuration'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/gem_runner.rb:33:in `run'
    from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/bin/gem:21:in `<main>'

It's not possible to run gem uninstall stringio or gem pristine stringio at this point - the only fix seems to be manually rm -rfing the gem directory.

The same isn't true for ruby from docker or rvm - I can update patch versions and still expect to load existing gems. Gems aren't compatible across minor/major version updates, but that's usually handled by GEM_PATH being scoped to the minor version (eg ~/.gem/ruby/3.1.0/, ~/.gem/ruby/3.2.0/, etc).

Steps To Reproduce

Steps to reproduce the behavior:

1. Clone nixpkgs, checkout 17fa5d3a36ec73c897f9024b7158e13867faf76b (where ruby_3_1 is at 3.1.3)
2. Use this version of ruby to install a gem with a c extension. Normally I'd have this in my shell and gem install foo would be installing to the default location of ~/.gem/ruby/3.1.0/gems, but as a self-contained one-liner:

   nix shell .#ruby_3_1 .#gnumake .#bintools  -c env GEM_HOME=./bundle GEM_PATH=./bundle LIBRARY_PATH=$(nix eval --raw .#glibc)/lib gem install stringio

3. Checkout 1aecb7bea358c820032072eca6f82f5a50efc52b (where ruby_3_1 is at 3.1.3)
4. Try and do anything with gem:

   $ nix shell .#ruby_3_1  -c env GEM_HOME=./bundle GEM_PATH=./bundle gem list
   <internal:/nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:88:in `require': incompatible library version - /root/nixpkgs/bundle/gems/stringio-3.0.5/lib/stringio.so (LoadError)
   from <internal:/nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:88:in `require'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych/nodes/node.rb:2:in `<top (required)>'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych/nodes.rb:2:in `require_relative'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych/nodes.rb:2:in `<top (required)>'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych.rb:15:in `require_relative'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/psych.rb:15:in `<top (required)>'
   from <internal:/nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:151:in `require'
   from <internal:/nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:151:in `require'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems.rb:608:in `load_yaml'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/config_file.rb:346:in `load_file'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/config_file.rb:189:in `initialize'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/gem_runner.rb:71:in `new'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/gem_runner.rb:71:in `do_configuration'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/lib/ruby/3.1.0/rubygems/gem_runner.rb:33:in `run'
   from /nix/store/yg797kp2fwwjzfzf78cf3ixqn7p5lid8-ruby-3.1.4/bin/gem:21:in `<main>'

Expected behavior

It successfully loads gems with c-extensions following patch-level upgrades.

Additional context

I think I've seen this even if the ruby patch level doesn't change after upgrading my nixpkgs channel, but haven't managed to narrow down a commit where that happens.

You can see this working better in docker with something like:

docker run -v $(pwd)/docker-bundle:/bundle -e GEM_PATH=/bundle -e GEM_HOME=/bundle ruby:3.1.2 gem install stringio
docker run -v $(pwd)/docker-bundle:/bundle -e GEM_PATH=/bundle -e GEM_HOME=/bundle ruby:3.1.3 gem list

I'm not sure what's special about the nix version of ruby that makes it so intolerant of patch-level updates.

Notify maintainers

@vrthra @manveru @marsam

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.105, NixOS, 22.11 (Raccoon), 22.11.3539.3e01f83884b`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.1`
 - channels(root): `"nixos-22.11"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[bittersweet]

I run into this same situation on every update as well. I keep uninstalling and reinstalling gems with native extensions which gets very tedious indeed, there are a lot :-) Would be great to find a definitive solution for this together.

[jdelStrother]

I've been mitigating it by overriding GEM_HOME to include nix's derivation hash, so my gems get installed to, eg, ~/.gem/ruby/ppaf93qzk1dx2zal81js8m91snfz48z8-ruby-3.1.4/ -

let gemHome = "$HOME/.gem/ruby/${builtins.baseNameOf ruby}"; in {

# ...

  shellHook = ''
    export GEM_HOME="${gemHome}";
    export GEM_PATH="${gemHome}";
  '';
}

[kirillrdy]

greetings ! @jdelStrother your solution is actually pretty neat !

The underlying issue is itself is pretty obvious, as non nix build systems have certain expectations about paths which don't hold true in nix

if you use gems from nix ( either via nixpkgs or bundix) you are not going to have those issues

here is example of using gems via bundix https://nixos.org/manual/nixpkgs/stable/#developing-with-ruby