search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.58k stars 13.74k forks source link

Build failure: zig_0_11 #299091

Open CrepeGoat opened 6 months ago

CrepeGoat commented 6 months ago

Steps To Reproduce

Steps to reproduce the behavior:

  1. run nix-shell -p zig

Build log

...
[100%] Building stage3
dyld: lazy symbol binding faileddyld: lazy symbol binding failed: Symbol not found: ___ulock_waidyld: lazy symbol binding failedt2
  Referenced from: /tmp/nix-b: Symbol not found: ___ulock_wai: Symbol not found: ___ulock_waiuild-zig-0.11.0.drv-0/source/buit2
  Referenced from: /tmp/nix-bt2
  Referenced from: /tmp/nix-bld/zig2
  Expected in: /usr/lib/uild-zig-0.11.0.drv-0/source/buiuild-zig-0.11.0.drv-0/source/builibSystem.B.dylib

ld/zig2
  Expected in: /usr/lib/ld/zig2
  Expected in: /usr/lib/libSystem.B.dylib

libSystem.B.dylib

dyld: Symbol not found: ___ulock_wait2
  Referenced from: /tmp/ndyld: Symbol not found: ___ulockix-build-zig-0.11.0.drv-0/source_wait2
  Referenced from: /tmp/ndyld: Symbol not found: ___ulock/build/zig2
  Expected in: /usr/ix-build-zig-0.11.0.drv-0/source_wait2
  Referenced from: /tmp/nlib/libSystem.B.dylib

/build/zig2
  Expected in: /usr/ix-build-zig-0.11.0.drv-0/sourcelib/libSystem.B.dylib

/build/zig2
  Expected in: /usr/lib/libSystem.B.dylib

make[2]: *** [CMakeFiles/stage3.dir/build.make:73: stage3/bin/zig] Abort trap: 6
make[1]: *** [CMakeFiles/Makefile2:196: CMakeFiles/stage3.dir/all] Error 2
make: *** [Makefile:136: all] Error 2```

full logs at https://gist.github.com/CrepeGoat/d4b5f983afd3268fafe175834bfb0bf0

Additional context

Add any other context about the problem here.

Notify maintainers

@andrewrk @AndersonTorres @figsoda

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"x86_64-darwin"`
 - host os: `Darwin 17.7.0, macOS 10.13.6`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.20.1`
 - channels(work): `"darwin"`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

-> this is a fairly old system. is it not supported? (my impression was that zig should be portable to ~all targets, as long as the target wasn't from the 1960's or something.)

Add a :+1: reaction to issues you find important.

iFreilicht commented 5 months ago

Can confirm, zig also doesn't build on aarch64-darwin, breaking many packages that are built with it.

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-darwin"`
 - host os: `Darwin 23.4.0, macOS 14.4.1`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.22.0pre20240320_7d2ead5`

Hydra has failed to build zig on darwin for three weeks now

iFreilicht commented 5 months ago

I get a different build error on current master, though:

44 warnings generated.
[ 94%] Linking CXX executable zig2
[ 94%] Built target zig2
[100%] Building stage3
error: unable to create compilation: DarwinSdkNotFound
make[2]: *** [CMakeFiles/stage3.dir/build.make:73: stage3/bin/zig] Error 1
make[1]: *** [CMakeFiles/Makefile2:196: CMakeFiles/stage3.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

EDIT: This seems to be the same issue as described here: https://github.com/NixOS/nixpkgs/issues/287861#issuecomment-1962225863

EDIT: Ok after further investigation, this seems to be a two part problem. Zig fails to build on darwin if sandboxing is enabled, which is bad, but a separate problem. My build error above was with sandbox = true. After setting it to false, running sudo launchctl kickstart -k system/org.nixos.nix-daemon, the build succeeds:

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-darwin"`
 - host os: `Darwin 23.4.0, macOS 14.4.1`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.22.0pre20240320_7d2ead5`
$ nix build --rebuild .#zig                                                                                                                                                                                                          ✔  6s  NORMAL 
error: derivation '/nix/store/11994gmkr3yh42vmhm7gj8zi3k7q8p8n-zig-0.11.0.drv' may not be deterministic: output '/nix/store/970nac7jhjaa28pimh7q103509vyknhy-zig-0.11.0' differs

Going further, I can now build the commit that was identified in https://github.com/NixOS/nixpkgs/issues/290512#issuecomment-1961034069 to be the last working one:

$ git checkout ac60eaff15a3a96bbec69d4316ab538f8f00d235
$ nix build --rebuild .#zig
error: derivation '/nix/store/9437b781qs7zr12sqf9pwismjlksm5bw-zig-0.11.0.drv' may not be deterministic: output '/nix/store/agiipadnw6bdhibwd5gcvyhf4292qyh8-zig-0.11.0' differs

And then build the next one that broke the build:

$ git checkout checkout 1d8c2f118c356995b8b171c8729f8592297a3319
$ git log --oneline | head -2
1d8c2f118c35 CoreFoundation: apply hook to correct platform
ac60eaff15a3 Merge pull request #284319 from NixOS/backport-284101-to-staging-23.11
$ nix build .#zig
....

Well, that's running now, I have rebuild stdenv for it. Will update when it's done. aborted, see below.

EDIT: While that is running, I've also tried to build zig on the commit that hydra failed for the first time, as that has most dependencies cached:

$ git worktree add ../nixpkgs-zigbroke 080a4a27
Preparing worktree (detached HEAD 080a4a27f206)
Updating files: 100% (40720/40720), done.
HEAD is now at 080a4a27f206 microsoft-edge: fix screen sharing on wayland
$ cd ../nixpkgs-zigbroke
$ nix build .#zig
$ result/bin/zig version
0.11.0
$ readlink -f result
/nix/store/xvw8ikyvk7jn2f3vj6xd9n1yxpgvk02r-zig-0.11.0

Hmm. This is very annoying. Zig not building in the sandbox means it's impure, so if the darwin build agents for hydra differ in their local setup from my machine, that could be the reason for the build failing.

I also built on ae10a6d, the commit of the latest eval, and the inputs to zig didn't change, so nix just used the already-built derivation.

EDIT: After looking at the hydra logs (should've done that before), the failure is precisely the one I got when sandbox was still enabled.

So we do need to fix the build in sandbox if we want to fix this build issue. The exact errors in the original post happened without sandbox, so was likely caused by the local setup.

wegank commented 5 months ago

Looks like Zig builds on darwin now? https://hydra.nixos.org/eval/1805832?filter=zig&compare=1805823&full=#tabs-still-succeed?

iFreilicht commented 5 months ago

Hmm, I don't know why that could be. Indeed, hydra built successfully, and nix build downloads from the cache, but trying to reproduce the result fails with the exact same error as before.

$ checkout 92d295f
$ nix build --print-out-paths --no-link .#zig
/nix/store/divv53p9wjrsqvi3g6v3c0vabvhcb3z3-zig-0.11.0
$ nix build --rebuild .#zig
error: builder for '/nix/store/93wiqvqj0xm7bw8difdzdixjxb0rx26y-zig-0.11.0.drv' failed with exit code 2;
       last 10 log lines:
       > static inline bool zig_subo_u64(uint64_t *res, uint64_t lhs, uint64_t rhs, uint8_t bits) {
       >                                           ^
       > 44 warnings generated.
       > [ 94%] Linking CXX executable zig2
       > [ 94%] Built target zig2
       > [100%] Building stage3
       > error: unable to create compilation: DarwinSdkNotFound
       > make[2]: *** [CMakeFiles/stage3.dir/build.make:73: stage3/bin/zig] Error 1
       > make[1]: *** [CMakeFiles/Makefile2:196: CMakeFiles/stage3.dir/all] Error 2
       > make: *** [Makefile:136: all] Error 2
       For full logs, run 'nix log /nix/store/93wiqvqj0xm7bw8difdzdixjxb0rx26y-zig-0.11.0.drv'.
$ nix-shell -p nix-info --run "nix-info -m" | grep sandbox
 - sandbox: `yes`

I also tried to check out the commit of the very first successful build that all the subsequent ones were cached from (3118b08635f5bd23d4daa1465571030683f38d1a), but that had the exact same result.

So it seems the build is very much not reproducible, meaning any change to zig's inputs might cause a rebuild that fails.

The derivation path on hydra (/nix/store/93wiqvqj0xm7bw8difdzdixjxb0rx26y-zig-0.11.0.drv) is also the exact same as the derivation that fails to build for me, so this is not caused by some stray files sitting in my repo.

There's some impurity here, but I can't say what it might be.

iFreilicht commented 5 months ago

Ok, it seems 0.12 builds successfully and reproducibly now, apparently there was an upstream fix for this. I opened a PR.

wegank commented 5 months ago

I'm not sure if this should be closed, since the issue still persists for Zig 0.11...

iFreilicht commented 5 months ago

Depending on how you look at it. Hydra itself built 0.11 (and 0.10 and 0.9) fine and they're cached, so for the majority of users, this issue is fixed.

zig <0.12 fails to reproduce on some (AFAICT newer) versions of macOS, but because this only affects rebuilds, only new macOS and has been fixed by upstream in 0.12, I don't see much reason to invest any time in trying to fix the old versions.

CrepeGoat commented 5 months ago

can confirm that nix-shell -p zig now works (and as mentioned installs zig 0.12.0), thanks @iFreilicht !

I'm not sure if this should be closed, since the issue still persists for Zig 0.11...

I have no comments here, just confirming that the linked PR seems to fix my particular issue.

nbraud commented 4 months ago

I'm not sure if this should be closed, since the issue still persists for Zig 0.11...

Probably not, then! Sorry, I didn't catch that the commit would close the issue.

mhanberg commented 3 months ago

I just updated my nixpkgs (unstable) and zig_0_11 gives the following failure

error: builder for '/nix/store/3w3k92kb3kj289x9d6h6zmmj3msv41ff-zig-0.11.0.drv' failed with exit code 2;
       last 10 log lines:
       > 1 error generated.
       > make[2]: *** [CMakeFiles/zigcpp.dir/build.make:118: CMakeFiles/zigcpp.dir/src/zig_clang_driver.cpp.o] Error 1
       > 1 error generated.
       > make[2]: *** [CMakeFiles/zigcpp.dir/build.make:76: CMakeFiles/zigcpp.dir/src/zig_llvm.cpp.o] Error 1
       > 1 error generated.
       > make[2]: *** [CMakeFiles/zigcpp.dir/build.make:104: CMakeFiles/zigcpp.dir/src/zig_clang.cpp.o] Error 1
       > 1 error generated.
       > make[2]: *** [CMakeFiles/zigcpp.dir/build.make:132: CMakeFiles/zigcpp.dir/src/zig_clang_cc1_main.cpp.o] Error 1
       > make[1]: *** [CMakeFiles/Makefile2:91: CMakeFiles/zigcpp.dir/all] Error 2
       > make: *** [Makefile:136: all] Error 2
       For full logs, run 'nix log /nix/store/3w3k92kb3kj289x9d6h6zmmj3msv41ff-zig-0.11.0.drv'.