Multi platform

[jdelStrother]

This builds on top of @lavoiesl's awesome work here https://github.com/nix-community/bundix/pull/68 - but hopefully gets rid of the main objections raised in that PR

By default bundix will generate more-or-less the same gemset as it always has, though each gem gets a couple of new keys to help with tracking: eg:

   nokogiri = {
     dependencies = ["mini_portile2" "racc"];
+    gem_platform = "ruby";
     groups = ["assets" "default" "development" "test"];
     platforms = [];
     source = {
       remotes = ["https://rubygems.org"];
       sha256 = "05rfzi8wksps5pgaavq1n1vkngsrjhqz8rcd1qdb52hnpg9q9p9b";
       type = "gem";
     };
+    target_platform = "ruby";
     version = "1.11.4";
   };

You can also specify a platform to generate the gemset for - eg bundix --platform=x86_64-darwin-19, which will fetch the native gems for x86_64-darwin-19 where available:

   nokogiri = {
-    dependencies = ["mini_portile2" "racc"];
+    dependencies = ["racc"];
+    gem_platform = "x86_64-darwin";
     groups = ["assets" "default" "development" "test"];
     platforms = [];
     source = {
       remotes = ["https://rubygems.org"];
-      sha256 = "05rfzi8wksps5pgaavq1n1vkngsrjhqz8rcd1qdb52hnpg9q9p9b";
+      sha256 = "0cyp98db8wpcp108w1b4iwh6ah98cwdydz2g2f5jlsig3qnf2m91";
       type = "gem";
     };
-    version = "1.11.4";
+    target_platform = "x86_64-darwin-19";
+    version = "1.11.4-x86_64-darwin";
   };

That's probably fine if all your developers are on the same platform, but if you need multiplatform support, there's also a new --platforms option. bundix --platforms=ruby,x86_64-darwin,x86_64-linux,aarch64-darwin generates 4 gemsets (gemset.nix, gemset.x86_64-darwin.nix, gemset.x86_64-linux.nix, & gemset.aarch64-darwin.nix), targetted at the different platforms.

Caveats:

* You need to run bundle lock --add-platform FOO before Bundix will look up the corresponding platform's gems. (Possibly the --platforms=ruby,x86_64-linux option should be replaced with, eg, --autoplatforms, which generates a gemset for each platform that's referenced in your Gemfile.lock ...)

• Subtle platform mismatches can lead to confusing bundler errors. I generated a gemset with bundix --platform=x86_64-darwin, which fetched x86_64-darwin-20 gems. Bundler then fails to load with "Could not find libv8-node-15.14.0.1 in any of the sources", because it tries to find the x86_64-darwin-19 version. Seems to be bundler-specific, though - loading the gem using a simple require works fine 😕 . And if I regenerate the gemset with bundix --platform=x86_64-darwin19, that also works fine - at least on x86_64-darwin-19 ruby...

• I'm not sure of a good nixy way of selecting which gemset to load in bundlerEnv. I was using this, till I ran into the above problem of needing to include the specific version number:

  pkgs.bundlerEnv {
  gemset = import ./gemset + ".${ruby.system}.nix";

[viraptor]

Confirming this works and solves issues with my platform-specific gems. Specifically if you use libv8-node without this patch, an arch version is chosen by default while running on x86-64 and fails the hash check. This PR's version works just fine.

Since there was no followup in this or base PR, I'm pinging maintainers again: @manveru @alyssais ?

[jsierles]

@jdelStrother Did you end up finding a way to select the correct gemset based on the platform?

[jdelStrother]

@jsierles For now I've been getting away with the regular bundix 2.5 release (plus a ruby3 fix), and this in my bundleEnv config:

pkgs.bundlerEnv {
  name = "myapp-gems";
  gemdir = ./.;
  # bundlerEnv doesn't play well with bundler's multiplatform support.
  # AFAICT it tries to load platform-specific gems that haven't been packaged.
  # Remove the platforms from Gemfile.lock and just use the 'ruby' platform.
  lockfile = pkgs.runCommand "platformless-gemfile" {} ''
    sed '/^PLATFORMS/,/^$/d' ${./Gemfile.lock} > $out
    echo -e "PLATFORMS\n  ruby" >> $out
  '';

which I've been using daily, though it's a year since I wrote it and I can't say I'm 100% confident in how everything fits together any more. Without it, I get errors like:

/nix/store/8f4ps3p3wdp0l41pf4w37l98ynl078nn-bundler-2.2.32/lib/ruby/gems/3.0.0/gems/bundler-2.2.32/lib/bundler/definition.rb:502:in `materialize': Could not find nokogiri-1.13.4-arm64-darwin in any of the sources (Bundler::GemNotFound)
    from /nix/store/8f4ps3p3wdp0l41pf4w37l98ynl078nn-bundler-2.2.32/lib/ruby/gems/3.0.0/gems/bundler-2.2.32/lib/bundler/definition.rb:239:in `specs_for'
    from /nix/store/8f4ps3p3wdp0l41pf4w37l98ynl078nn-bundler-2.2.32/lib/ruby/gems/3.0.0/gems/bundler-2.2.32/lib/bundler/runtime.rb:18:in `setup'
    from /nix/store/8f4ps3p3wdp0l41pf4w37l98ynl078nn-bundler-2.2.32/lib/ruby/gems/3.0.0/gems/bundler-2.2.32/lib/bundler.rb:152:in `setup'
    from /nix/store/9bwmv3plhi85nb6fj4hsmhjhbhzkmzw4-myappp-gems/bin/rails:19:in `<main>'

[jsierles]

Ah, that's a good trick! I was removing platform-specific gems from the Gemfile.lock manually, but they would end up getting added back by non-Nix users.

[1passenger]

I had similiar issues, this line solved it for me:

bundler config set force_ruby_platform true --local

[aidalgol]

Is bundix the right place to address this, or should bundler in nixpkgs be patched so that the force_ruby_platform config flag is true by default?

[will]

Unfortunately the force platform trick doesn’t work for the sorbet gemOn Jun 26, 2023, at 07:41, Aidan Gauland @.***> wrote: Is bundix the right place to address this, or should bundler in nixpkgs be patched so that the force_ruby_platform config flag is true by default?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>