search

nix-community / bundix

Generates a Nix expression for your Bundler-managed application. [maintainer=@manveru]
163 stars 55 forks source link

bundix doesn't like architecture-specific gems #71

Open endgame opened 4 years ago

endgame commented 4 years ago

I am trying to bundix the sorbet typechecker.

Gemfile:

source 'https://rubygems.org'
gem 'sorbet'

default.nix (I use callPackage on this):

{ bundlerApp, lib }:

bundlerApp rec {
  pname = "sorbet";
  gemdir = ./.;
  exes = [ "sorbet" "sorbet-static" ];

  meta = with lib; {
    description = "Static type checker for ruby";
    homepage = "https://sorbet.org";
    license = licenses.asl20;
    maintainers = [ maintainers.endgame ];
    platforms = platforms.unix;
  };
}

Because of #70 , I am trying to generate gemset.nix using bundle update && bundix. Here is what happens:

First, bundler correctly downloads the gem that matches my arch (x86_64-linux):

$ bundle update
Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using bundler 2.1.4
Fetching sorbet-static 0.5.5549 (x86_64-linux)
Installing sorbet-static 0.5.5549 (x86_64-linux)
Fetching sorbet 0.5.5549
Installing sorbet 0.5.5549
Bundle updated!
Post-install message from sorbet:

  Thanks for installing Sorbet! To use it in your project, first run:

    bundle exec srb init

  which will get your project ready to use with Sorbet.
  After that whenever you want to typecheck your code, run:

    bundle exec srb tc

  For more docs see: https://sorbet.org/docs/adopting

Then bundix downloads a Darwin gem when constructing gemset.nix?

$ bundix
[4.7 MiB DL]
path is '/nix/store/8ycbkwk3v17azzml1836r9lhjdi6l03b-sorbet-static-0.5.5549-universal-darwin-14.gem'
1vli775jl3zkb39mqj26nwk2y8lzc7hv8h8cs4yilnf6fndzab4p => sorbet-static-0.5.5549-universal-darwin-14.gem
[0.0 MiB DL]
path is '/nix/store/spyfvfppa7z08595vqwsm6p0ya8x1aza-sorbet-0.5.5549.gem'
02c09sh99nk4hhf1k768lz7vq3l759h083n6v1cjkmqha1p8b0a1 => sorbet-0.5.5549.gem

Attempting to build this fails:

$ nix-build -E '(import <nixpkgs> {}).callPackage ./. {}'
these derivations will be built:
  /nix/store/xqpqakw2cpg7hsvm6mm5xib9xrlafkr8-sorbet-static-0.5.5549.gem.drv
  /nix/store/53iklkb8qsf1imbffbvpnwsg2nxpn8qd-ruby2.6.6-sorbet-static-0.5.5549.drv
  /nix/store/qmdhdjnrdhr55xr9gvqgsmdg3hvwzxkp-ruby2.6.6-sorbet-0.5.5549.drv
  /nix/store/gzkav0w1d4axhqwmy843y2f1ci81ja2s-sorbet-0.5.5549.drv
  /nix/store/265y5dnrcaykcajwm0z3s5829agb2pyf-sorbet-0.5.5549.drv
building '/nix/store/xqpqakw2cpg7hsvm6mm5xib9xrlafkr8-sorbet-static-0.5.5549.gem.drv'...

trying https://rubygems.org/gems/sorbet-static-0.5.5549.gem
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 403 Forbidden
error: cannot download sorbet-static-0.5.5549.gem from any mirror
builder for '/nix/store/xqpqakw2cpg7hsvm6mm5xib9xrlafkr8-sorbet-static-0.5.5549.gem.drv' failed with exit code 1
cannot build derivation '/nix/store/53iklkb8qsf1imbffbvpnwsg2nxpn8qd-ruby2.6.6-sorbet-static-0.5.5549.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/gzkav0w1d4axhqwmy843y2f1ci81ja2s-sorbet-0.5.5549.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/265y5dnrcaykcajwm0z3s5829agb2pyf-sorbet-0.5.5549.drv': 1 dependencies couldn't be built
error: build of '/nix/store/265y5dnrcaykcajwm0z3s5829agb2pyf-sorbet-0.5.5549.drv' failed

Note that it's trying to download a gem file with no architecture. If you visit the gem page for sorbet-static, you'll see a download link that includes the arch. This seems to be getting lost somewhere. Not sure if that is a bundlerApp bug in nix code, or a problem with the generated gemset.nix. But I'm pretty sure that bundix looking at the Darwin gem when constructing gemset.nix is wrong.

reedlaw commented 4 years ago

I have the same issue. It's trying to download from a url that doesn't exist (missing _x86_64-linux suffix).

This is the correct path:

https://rubygems.org/downloads/sorbet-static-0.5.5713-x86_64-linux.gem

It's trying to fetch this:

https://rubygems.org/downloads/sorbet-static-0.5.5713.gem
reedlaw commented 4 years ago

I found an open PR for this.

sir4ur0n commented 2 years ago

In the meantime, for those who would like to use Sorbet with Nix and Linux, the following works:

let
  # Import Nixpkgs however you like, be it <nixpkgs> or with Niv
  pkgs = ...;

  rubyGems = pkgs.bundlerEnv {
    name = "gems-for-my-project";
    ruby = pkgs.ruby_3_1;
    gemdir = ./path/to/gemfile/directory;
    gemConfig = pkgs.defaultGemConfig // {
      # Poor man's solution: `sorbet-static` has a specific version per system, and Bundix currently does not support that.
      # Additionally, the `sorbet` executable needs ELF patching.
      # If the Sorbet version ever changes (after re-running `bundle update` then `bundix`) then the SHA256 must be updated accordingly.
      # The simplest way to do so is to leave it empty and look at the error message to get the right hash.
      # See https://nixos.org/manual/nixpkgs/stable/#developing-with-ruby
      # See https://github.com/nix-community/bundix/issues/71
      sorbet-static = attrs:
        # Bundix picks the Darwin version by default, so we only need to patch on Linux
        if pkgs.stdenv.isLinux then {
          # Append the OS-specific version suffix
          version = attrs.version + "-x86_64-linux";
          # Update the SHA256 accordingly
          source = attrs.source // { sha256 = "sha256-nuvwK1pQSJ4EsYdkT42q5NsG4QlZZuYT9Zxzz6839rk="; };
          # Post-fix the gem by fixing the interpreter with the correct Nix one (default interpreter is `/lib64/ld-linux-x86-64.so.2` which usually does not exist on Nix machines)
          nativeBuildInputs = with pkgs; [ autoPatchelfHook ];
        } else attrs;
    };
  };
in
pkgs.mkShell {
  buildInputs = [ 
      # Bundler must be before `rubyGems` because the `bundle` executable provided by `rubyGems` does not allow updating the `Gemfile.lock` files
      bundler
      rubyGems.wrappedRuby
      rubyGems
      bundix
 ];
}

The SHA256 part is not ideal, but otherwise it seems to work just fine.

Also, I think the code above can be easily adapted to other gems than Sorbet with a similar architecture-specific issue.

I hope this is helpful to someone!

nixos-discourse commented 2 years ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/issues-with-nix-reproducibility-on-macos-trying-to-build-nokogiri-ruby-error-unknown-warning-option/22019/8

will commented 1 year ago

Thanks @sir4ur0n this helped me a lot to figure out how to use autoPatchelfHook on the sorbet gem. I vendored a forked version of bundix @jdelStrother's patches to generate separate gemsets as well as support ruby 3, and this is my current thing that is so far working great for linux and arm mac sorbet 

      let
        overlays = [
          (self: super: {
            ruby = pkgs.ruby_3_1;
          })
        ];
        pkgs = import nixpkgs { inherit overlays system; };

        bundlerPlatform = if pkgs.ruby.system == utils.system.aarch64-darwin then "arm64-darwin" else pkgs.ruby.system;
        rubyEnv = pkgs.bundlerEnv {
          name = "ruby-env";
          inherit (pkgs) ruby;
          gemdir = ./.;
          gemset = .nix/gemset + ".${bundlerPlatform}.nix";
          gemConfig = pkgs.defaultGemConfig // {
            sorbet-static = attrs:
              if pkgs.stdenv.isLinux then {
                nativeBuildInputs = with pkgs; [ autoPatchelfHook ];
              } else attrs;
          };
        };