NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.06k stars 14.11k forks source link

rabbitmq-server doesn't build on x86_64-darwin #169956

Open turion opened 2 years ago

turion commented 2 years ago

In https://github.com/NixOS/nixpkgs/pull/169684 we noticed that rabbitmq-server doesn't build on x86_64-darwin anymore despite no changes relevant to this.

This is probably due to a cross-compilation issue in erlang on darwin. @reckenrode did some valuable research on this: https://github.com/NixOS/nixpkgs/pull/169684#issuecomment-1107470733

Probably, elixir and erlang need to go to depsBuildBuild, but this doesn't fix the problem entirely.

CC @armeenm @NixOS/darwin-maintainers

thefloweringash commented 2 years ago

I think the underlying problem is that x86_64-darwin erlang doesn't work under Rosetta 2, which means it's probably an upstream bug.

# on an M1 Mac Mini
❯ $(nix-build -A erlang --argstr system x86_64-darwin)/bin/erl
zsh: segmentation fault  $(nix-build -A erlang --argstr system x86_64-darwin)/bin/erl
With lldb ``` ❯ ERLANG=$(nix-build -A erlang --argstr system x86_64-darwin); export ROOTDIR="$ERLANG/lib/erlang"; export BINDIR="$ROOTDIR/erts-12.2/bin"; lldb $BINDIR/beam.smp (lldb) target create "/nix/store/h7krx0a1if4wy562p2h9n8srr3krclnf-erlang-24.2/lib/erlang/erts-12.2/bin/beam.smp" Current executable set to '/nix/store/h7krx0a1if4wy562p2h9n8srr3krclnf-erlang-24.2/lib/erlang/erts-12.2/bin/beam.smp' (x86_64). (lldb) run Process 21342 launched: '/nix/store/h7krx0a1if4wy562p2h9n8srr3krclnf-erlang-24.2/lib/erlang/erts-12.2/bin/beam.smp' (x86_64) Process 21342 stopped * thread #6, name = '2_scheduler', stop reason = EXC_BAD_ACCESS (code=1, address=0x58) frame #0: 0x000000010ac00504 -> 0x10ac00504: movq 0x10(%rsi), %rcx 0x10ac00508: movq 0x18(%rsi), %r8 0x10ac0050c: movq 0x20(%rsi), %r9 0x10ac00510: callq 0x1000248b0 ; beam_jit_call_nif Target 0: (beam.smp) stopped. (lldb) bt * thread #6, name = '2_scheduler', stop reason = EXC_BAD_ACCESS (code=1, address=0x58) * frame #0: 0x000000010ac00504 ```

Nixpkgs cross and Rossetta 2 are orthogonal systems. Setting --argstr system x86_64-darwin means that build = host = target = x86_64-darwin and all dependencies should be from the same package set, so in this case it shouldn't make any difference where dependencies go (at least with strictDeps = false).

Regarding https://github.com/NixOS/nixpkgs/pull/169684#issuecomment-1107470733

Interestingly, erlang is listed under buildInputs, but it appears to be pulling in the aarch64-darwin version even when I specify --system x86_64-darwin. The path to erl is:

  • /nix/store/nvngkn9yj4gx1b5jxfallzrxk2vi7l21-erlang-24.2/bin/erl

I'm suspicious of this result:

❯ nix-shell -A rabbitmq-server --argstr system x86_64-darwin --run 'command -v erl'
/nix/store/h7krx0a1if4wy562p2h9n8srr3krclnf-erlang-24.2/bin/erl
❯ nix-shell -A rabbitmq-server --argstr system aarch64-darwin --run 'command -v erl'
/nix/store/nvngkn9yj4gx1b5jxfallzrxk2vi7l21-erlang-24.2/bin/erl

Probably, elixir and erlang need to go to depsBuildBuild, but this doesn't fix the problem entirely.

For nixpkgs cross compilation, this is likely correct.

turion commented 2 years ago

Does it surprise you that Hydra was green on this job recently? At least this is how I read it: https://hydra.nixos.org/build/174137098

thefloweringash commented 2 years ago

Does it surprise you that Hydra was green on this job recently? At least this is how I read it: https://hydra.nixos.org/build/174137098

If that's a question for me, then no, not at all. The suspected failure is x86_64-darwin erlang running under Rosetta 2, but the machine that built rabbitmq-server (f402c2c2) is native x86_64-darwin. (Slight assumption on my part, based on the hydra machine status page showing it only supporting x86_64-darwin, and having a different naming scheme to the aarch64-darwin machines).