Bundix does not work: "Could not find 'bundler'"

[chris-martin]

Describe the bug Bundix, and thus the Ruby infrastructure, does not work.

To Reproduce Steps to reproduce the behavior:

1. Run the command $(nix-build '<nixpkgs>' -A bundix --no-out-link)/bin/bundix --magic given in the manual

Expected behavior I'm not sure, I haven't gotten any farther than trying to run the command that the manual says to run.

Screenshots

$ $(nix-build '<nixpkgs>' -A bundix --no-out-link)/bin/bundix --magic
Traceback (most recent call last):
        8: from /nix/store/5ngg2yc4m5q0l98yqx2lwy261fx6rzn7-bundix-2.4.1/bin/.bundix-wrapped:18:in `<main>'
        7: from /nix/store/abd2zgz6vvmf9gxw5ra49ibqp1gs15c3-ruby-2.5.5/lib/ruby/2.5.0/rubygems.rb:304:in `activate_bin_path'
        6: from /nix/store/abd2zgz6vvmf9gxw5ra49ibqp1gs15c3-ruby-2.5.5/lib/ruby/2.5.0/rubygems.rb:304:in `synchronize'
        5: from /nix/store/abd2zgz6vvmf9gxw5ra49ibqp1gs15c3-ruby-2.5.5/lib/ruby/2.5.0/rubygems.rb:305:in `block in activate_bin_path'
        4: from /nix/store/abd2zgz6vvmf9gxw5ra49ibqp1gs15c3-ruby-2.5.5/lib/ruby/2.5.0/rubygems/specification.rb:1440:in `activate'
        3: from /nix/store/abd2zgz6vvmf9gxw5ra49ibqp1gs15c3-ruby-2.5.5/lib/ruby/2.5.0/rubygems/specification.rb:1458:in `activate_dependencies'
        2: from /nix/store/abd2zgz6vvmf9gxw5ra49ibqp1gs15c3-ruby-2.5.5/lib/ruby/2.5.0/rubygems/specification.rb:1458:in `each'
        1: from /nix/store/abd2zgz6vvmf9gxw5ra49ibqp1gs15c3-ruby-2.5.5/lib/ruby/2.5.0/rubygems/specification.rb:1469:in `block in activate_dependencies'
/nix/store/abd2zgz6vvmf9gxw5ra49ibqp1gs15c3-ruby-2.5.5/lib/ruby/2.5.0/rubygems/dependency.rb:312:in `to_specs': Could not find 'bundler' (2.0.1) required by your /home/chris/git/lobsters/Gemfile.lock. (Gem::MissingSpecVersionError)
To update to the latest version installed on your system, run `bundle update --bundler`.
To install the missing version, run `gem install bundler:2.0.1`
Checked in 'GEM_PATH=/nix/store/4j8nrml4hnz5ic4xy3immgd0r62dqqsp-bundler-1.17.2/lib/ruby/gems/2.5.0:/nix/store/5ngg2yc4m5q0l98yqx2lwy261fx6rzn7-bundix-2.4.1/lib/ruby/gems/2.5.0:/nix/store/abd2zgz6vvmf9gxw5ra49ibqp1gs15c3-ruby-2.5.5/lib/ruby/gems/2.5.0', execute `gem env` for more information

Additional context I'm on the nixos-19.03 channel, and the project I'm trying to package is lobsters.

Metadata Please run nix run nixpkgs.nix-info -c nix-info -m and paste the result.

$ nix run nixpkgs.nix-info -c nix-info -m
error: attribute 'nixpkgs' in selection path 'nixpkgs.nix-info' not found

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[aanderse]

cc @manveru

[marsam]

I think it's because the Gemfile.lock was created with bundler2, which currently is not in nixpkgs. Can you try the following?

diff --git i/Gemfile.lock w/Gemfile.lock
index 948509e..48c1902 100644
--- i/Gemfile.lock
+++ w/Gemfile.lock
@@ -280,4 +280,4 @@ DEPENDENCIES
   unicorn

 BUNDLED WITH
-   2.0.1
+   1.17.2

[lrworth]

Seems to me that bundler2 now comes with ruby.

bash-3.2$ which bundler
/nix/store/axmaq9if5bmffibr0lng7hkhn1hxqvl9-ruby-2.6.5/bin/bundler
bash-3.2$ bundler --version
Bundler version 2.1.4

[SuppenGeist]

I still have the same problem even with ruby 2.6.5. Setting BUNDLED WITH to 1.17.2 fixes it.

[tbenst]

Just hit this. anyone know a workaround? Not sure how to use github-pages with nix if can't install from gemfile

[aanderse]

I thought this was fixed... @manveru do you know anyone else actively maintaining ruby infrastructure in nixpkgs that we should ping?

[manveru]

The "workaround" is to use bundix and bundler from the same nixpkgs version.

When it complains about old version, just re-generate the Gemfile.lock by running bundle lock which will then change the BUNDLED WITH to the matching one. You can also just delete the version, or change it manually to the version you need.

I'm not sure what we can do on our side to make this more clear... This is mostly an annoying habit of bundler and we cannot circumvent that without patching its Gemfile.lock generation directly.

So bundler upgrades in nixpkgs remain annoying, but are neccesary.

[tbenst]

I'm on 20.03, and generating a lockfile with nix's bundler:

> nix-shell --pure -p "ruby.withPackages(ps: with ps; [ jekyll bundler ])" -p bundix

$ bundle lock
Ignoring commonmarker-0.17.13 because its extensions are not built. Try: gem pristine commonmarker --version 0.17.13
Ignoring nokogiri-1.10.9 because its extensions are not built. Try: gem pristine nokogiri --version 1.10.9
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...
Writing lockfile to /home/tyler/code/savebrowntrack.github.io/Gemfile.lock

$ bundix
Traceback (most recent call last):
        8: from /nix/store/pb6z8zf0ghxw2scwyphmx9k5qzj4cgks-bundix-2.5.0/bin/.bundix-wrapped:18:in `<main>'
        7: from /nix/store/7pv5x751g4c1p06hn038iggsgmxzac28-ruby-2.6.5/lib/ruby/2.6.0/rubygems.rb:304:in `activate_bin_path'
        6: from /nix/store/7pv5x751g4c1p06hn038iggsgmxzac28-ruby-2.6.5/lib/ruby/2.6.0/rubygems.rb:304:in `synchronize'
        5: from /nix/store/7pv5x751g4c1p06hn038iggsgmxzac28-ruby-2.6.5/lib/ruby/2.6.0/rubygems.rb:305:in `block in activate_bin_path'
        4: from /nix/store/7pv5x751g4c1p06hn038iggsgmxzac28-ruby-2.6.5/lib/ruby/2.6.0/rubygems/specification.rb:1417:in `activate'
        3: from /nix/store/7pv5x751g4c1p06hn038iggsgmxzac28-ruby-2.6.5/lib/ruby/2.6.0/rubygems/specification.rb:1435:in `activate_dependencies'
        2: from /nix/store/7pv5x751g4c1p06hn038iggsgmxzac28-ruby-2.6.5/lib/ruby/2.6.0/rubygems/specification.rb:1435:in `each'
        1: from /nix/store/7pv5x751g4c1p06hn038iggsgmxzac28-ruby-2.6.5/lib/ruby/2.6.0/rubygems/specification.rb:1446:in `block in activate_dependencies'
/nix/store/7pv5x751g4c1p06hn038iggsgmxzac28-ruby-2.6.5/lib/ruby/2.6.0/rubygems/dependency.rb:313:in `to_specs': Could not find 'bundler' (2.1.4) required by your /home/tyler/code/savebrowntrack.github.io/Gemfile.lock. (Gem::MissingSpecVersionError)
To update to the latest version installed on your system, run `bundle update --bundler`.
To install the missing version, run `gem install bundler:2.1.4`
Checked in 'GEM_PATH=/nix/store/80xrd6npj2c8nsq22gr9jkvyppj1f7my-bundler-1.17.3/lib/ruby/gems/2.6.0:/nix/store/pb6z8zf0ghxw2scwyphmx9k5qzj4cgks-bundix-2.5.0/lib/ruby/gems/2.6.0:/nix/store/7pv5x751g4c1p06hn038iggsgmxzac28-ruby-2.6.5/lib/ruby/gems/2.6.0', execute `gem env` for more information

[stale[bot]]

I marked this as stale due to inactivity. → More info

[drzln]

I am hitting this. My local bundler is and should be 1.17.3

[luis@rio:~/code/tapresearch/raynor]$ bundler --version
Bundler version 1.17.3

[luis@rio:~/code/tapresearch/raynor]$

however bundix complains it cannot find it

Traceback (most recent call last):
        8: from /nix/store/95z7ajqpq9wld4i0x11qijjrs2q8hak6-bundix-2.5.1/bin/.bundix-wrapped:18:in `<main>'
        7: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems.rb:298:in `activate_bin_path'
        6: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems.rb:298:in `synchronize'
        5: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems.rb:299:in `block in activate_bin_path'
        4: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/specification.rb:1372:in `activate'
        3: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/specification.rb:1390:in `activate_dependencies'
        2: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/specification.rb:1390:in `each'
        1: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/specification.rb:1402:in `block in activate_dependencies'
/nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/dependency.rb:313:in `to_specs': Could not find 'bundler' (1.17.3) required by your /home/luis/code/tapresearch/raynor/Gemfile.lock. (Gem::MissingSpecVersionError)
To update to the latest version installed on your system, run `bundle update --bundler`.
To install the missing version, run `gem install bundler:1.17.3`
Checked in 'GEM_PATH=/nix/store/mz69s3mfhjmakvcf7438rg14pa2vfm8q-bundler-2.3.9/lib/ruby/gems/2.7.0:/nix/store/95z7ajqpq9wld4i0x11qijjrs2q8hak6-bundix-2.5.1/lib/ruby/gems/2.7.0:lib/ruby/gems/2.7.0' , execute `gem env` for more information
        8: from /nix/store/95z7ajqpq9wld4i0x11qijjrs2q8hak6-bundix-2.5.1/bin/.bundix-wrapped:18:in `<main>'
        7: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems.rb:298:in `activate_bin_path'
        6: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems.rb:298:in `synchronize'
        5: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems.rb:299:in `block in activate_bin_path'
        4: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/specification.rb:1372:in `activate'
        3: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/specification.rb:1390:in `activate_dependencies'
        2: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/specification.rb:1390:in `each'
        1: from /nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/specification.rb:1401:in `block in activate_dependencies'
/nix/store/7hc78l1iq9ibjc59qyfzb2fm2pi04m1h-ruby-2.7.6/lib/ruby/2.7.0/rubygems/specification.rb:1404:in `rescue in block in activate_dependencies': Could not find 'bundler' (>= 1.11) among 51 total gem(s) (Gem::MissingSpecError)
Checked in 'GEM_PATH=/nix/store/mz69s3mfhjmakvcf7438rg14pa2vfm8q-bundler-2.3.9/lib/ruby/gems/2.7.0:/nix/store/95z7ajqpq9wld4i0x11qijjrs2q8hak6-bundix-2.5.1/lib/ruby/gems/2.7.0:lib/ruby/gems/2.7.0' at: /nix/store/95z7ajqpq9wld4i0x11qijjrs2q8hak6-bundix-2.5.1/lib/ruby/gems/2.7.0/specifications/bundix-2.5.0.gemspec, execute `gem env` for more information

Current packaging that causes this is...

{ stdenv, bundlerEnv, buildRubyGem, pkgs, lib, }:
with pkgs;
let

  # a ruby interpreter for building other things
  # bundler 1.17.3
  # minitest 5.10.1
  build_ruby = stdenv.mkDerivation {
    buildInputs = [ zlib openssl ];
    name = "ruby-2.4";
    gemPath = "./.gem";
    meta = { platforms = [ "x86_64-linux" ]; };
    src = fetchurl {
      url = "https://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.4.tar.gz";
      sha256 = "sha256-JU8cGnnkzIFNHnMgvFvdmV3FfghyfTCnZ2ZGGanIrlo=";
    };
  };

  this_bundler = buildRubyGem
    rec {
      ruby = build_ruby;
      name = "${gemName}-${version}";
      gemName = "bundler";
      version = "1.17.3";
      source.sha256 = "sha256-vEv3W1SLJ0UaqfRDsYxGpzndIq1596X5C0hTdqZ9w1I=";
      dontPatchShebangs = true;
      postFixup = ''                                                        
        sed -i -e "s/activate_bin_path/bin_path/g" $out/bin/bundle          
    '';
    };

  ruby_simple = stdenv.mkDerivation {
    buildInputs = [ zlib this_bundler ];
    name = "ruby-2.4";
    gemPath = "./.";
    meta = { platforms = [ "x86_64-linux" ]; };
    src = fetchurl {
      url = "https://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.4.tar.gz";
      sha256 = "sha256-JU8cGnnkzIFNHnMgvFvdmV3FfghyfTCnZ2ZGGanIrlo=";
    };
  };

  # there is a script in the raynor repository called
  # ./bin/bundler this should have priority over in the
  # path.  We should not remove ./bin/bundler since it
  # might be used and I don't think nix configs
  # should need to effect anything in a repository
  # so it is reasonable to create a shell script and pass
  # it to the bundler in the derivation.
  bundle_script = pkgs.writeShellScriptBin "bundler" ''
    ${this_bundler}/bin/bundler $@
  '';

  # ruby = ruby_2_4.overrideAttrs (attrs: rec {
  #   patches = [];
  #   # phases = ["installPhase"];
  #   # pname = "ruby";
  #   # version = "2.4.4";
  #   # NROFF = "";
  #   # outputs = ["out"];
  #   # nativeBuildInputs = [];
  #   # buildInputs = [];
  #   # enableParallelBuilding = true;
  #   # patches = [];
  #   # installFlags = "";
  #   # # installPhase = ''
  #   # #   mkdir -p $out
  #   # #   autoconf -f
  #   # #   ./configure --prefix=$out
  #   # #   make clean
  #   # #   make
  #   # #   make install
  #   # #   make clean
  #   # # '';
  #   # src = fetchurl {
  #   #   url = "https://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.4.tar.gz";
  #   #   sha256 = "sha256-JU8cGnnkzIFNHnMgvFvdmV3FfghyfTCnZ2ZGGanIrlo=";
  #   # };
  # });
  minitest = buildRubyGem {
    inherit ruby;
    gemName = "minitest";
    type = "gem";
    version = "5.10.1";
    source.sha256 = "1yk2m8sp0p5m1niawa3ncg157a4i0594cg7z91rzjxv963rzrwab";
    gemPath = [ ];
  };
  this_bundix = bundix.overrideAttrs (attrs: rec {
    buildInputs = [ this_bundler build_ruby minitest rake nix-prefetch-scripts ];
  });

  gems = bundlerEnv {
    name = "rubyproject";
    gemdir = ./.;
    ruby = ruby_simple;
    # gemConfig = pkgs.defaultGemConfig // {
    #   nokogiri = attrs: {
    #     buildFlags = [
    #       "--use-system-libraries"
    #       "--with-zlib-lib=${zlib.out}/lib"
    #       "--with-zlib-include=${zlib.dev}/include"
    #       "--with-xml2-lib=${libxml2.out}/lib"
    #       "--with-xml2-include=${libxml2.dev}/include/libxml2"
    #       "--with-xslt-lib=${libxslt.out}/lib"
    #       "--with-xslt-include=${libxslt.dev}/include"
    #       "--with-exslt-lib=${libxslt.out}/lib"
    #       "--with-exslt-include=${libxslt.dev}/include"
    #       "--with-iconv-dir=${libiconv}"
    #       "--with-opt-include=${libiconv}/include"
    #     ];
    #     BuildInputs = with pkgs; [
    #
    #       libxslt
    #       libxslt.dev
    #
    #       libxml2
    #       libxml2.dev
    #
    #       zlib
    #       zlib.dev
    #
    #       libiconv
    #     ];
    #   };
    # };
  };
in
stdenv.mkDerivation {
  name = "rubyproject";
  src = ./.;
  # this way we can get nokogiri
  # buildInputs = [ gems.wrappedRuby gems ruby ];
  buildInputs = [ gems.wrappedRuby gems this_bundler bundle_script this_bundix ];
  # dontUnpack = true;
  shellHook = ''
    # prioritize ${this_bundler} being called over ./bin/bundler
    PATH=${this_bundler}/bin:$PATH
  '';
  installPhase = ''
    cp -r $src $out
  '';
}

it attempts to use a derivation of bundler-2.3.9 when I would like it to target the bundler-1.17.3 that exists.