NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.21k stars 14.21k forks source link

Tracking issue for Darwin stdenv LLVM update #234710

Closed reckenrode closed 1 year ago

reckenrode commented 1 year ago

This is a tracking issue to update the version of LLVM in the Darwin stdenv. I am breaking this out of #229786 into a tracking issue so I can submit and track PRs for the work I have done while I work through the ofborg outPath check failures. The longer I wait, the greater the risk of merge conflicts.

This issue was prompted by #229210 because the 13.3 SDK requires a newer Clang than the Darwin stdenv provides currently. To facilitate easier LLVM updates in the future, I have created a PR to rework the Darwin stdenv to decouple it from the versions of programs included in the bootstrap tools. I have also reworked it to follow the patterns used in the Linux stdenv.

See the comments in PR explaining the rework. My hope is it will make maintenance easier and more approachable. It should also make experimentation a little safer due to evaluation-time failures that stop “obvious” mistakes from turning into wasted build time. Possible areas for future exploration: removing ICU (in favor of the icu in nixpkgs) and libiconv (in favor of libiconvReal), using LTO during bootstrap.

LLVM Update PR

Previous PR

Rework Prerequisite PRs

These PRs are required for the stdenv bump. They reflect breakage due to stricter checks by clang or as required by the stdenv rework. Most of these PRs are required for clang 15. Some are required for clang 16. They should work with clang 11.

This PR was dropped.

LLVM Bump Prerequisite PRs

These PRs are not required for the rework, but they are required prior to the bump because it prevents the bootstrap from completing with clang 16.

Related PRs

These fail to build due to the changes or pertain to some other issue uncovered while working on the bump. They are not required for the bump, but they’ll make it go more smoothly after it is merged.

These PRs were dropped.

These are Darwin sandbox-related fixes. They’re not strictly necessary, but they allow Darwin users to build more things with the sandbox enabled.

This fix is related to running the store on case-sensitive APFS. It’s the first thing that has broken for me. Most case sensitivity issues during builds are due to /tmp being insensitive.

This issue is SIP-related, which shouldn’t affect most users.

reckenrode commented 1 year ago

Things are looking pretty good, but alas I have another round of PRs that I’ll be opening up soon. Some of them are required for the stdenv rework while others are required for the actual bump. My current thinking is to bump to LLVM 16 because that will match what Apple will ship with Xcode 15. However, that requires fixing a number of packages to build with clang 16, which is more strict about function prototypes and some other stuff. Those are the additional PRs I will be submitting.

The only potential blocker with LLVM 16 is clang 16 sometimes fails with a bad file descriptor error. I think it has to do with the use of process substitution to create the response file used to pass arguments to clang in cc-wrapper. It usually succeeds if you run it again, so reproducing to debug and determine where the fix needs to go has been challenging.

eliasnaur commented 1 year ago

@reckenrode it's a long shot, but is https://github.com/NixOS/nix/issues/6516 related to your bad file descriptor error? Only relevant if you're testing in CA mode.

reckenrode commented 1 year ago

@reckenrode it's a long shot, but is NixOS/nix#6516 related to your bad file descriptor error? Only relevant if you're testing in CA mode.

Unfortunately, probably not. CA mode doesn’t even work on aarch64-darwin. #178280 might be related. I haven’t really dug into it too deeply because I can restart the build when it happens. That’s not good for a final PR, but it lets me get everything else building before I try to dig into root causes. The big difference for me compared to NixOS/nix#6516 is it’s always the same FD (63). Bash’s process substitution starts counting at 63, which is why I believe cc-wrapper is involved. The question is why the FD doesn’t live long enough and what is trying to read it once it’s gone.

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/darwin-again/29331/14

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/darwin-again/29331/16

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-macos-monthly/12330/40

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-macos-monthly/12330/41

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/macos-system-integrity-protection-and-dy-dl-library-path/31729/5

viraptor commented 1 year ago

Linked some fix PRs ^ found from my home-manager failing to build from the branch. I'll do another batch soon.

tobim commented 1 year ago

Full report at https://gist.github.com/tobim/5ce3c6d6f49f29ac43b3ad76b96bd1e4.

Problematic dependencies

namecount
x86_64-darwin luajit2.1-mpack-1.0.9-0 3209
x86_64-darwin python3.10-pybind11-2.11.1 1750
x86_64-darwin python3.11-pybind11-2.11.1 1645
x86_64-darwin nodejs-18.17.1 691
x86_64-darwin python3.10-ruamel-yaml-clib-0.2.7 679
x86_64-darwin python3.11-ruamel-yaml-clib-0.2.7 491
x86_64-darwin perl5.38.0-IO-Tty-1.15 463
x86_64-darwin guile-2.2.7 426
x86_64-darwin grpc-1.57.0 407
x86_64-darwin perl5.36.1-IO-Tty-1.15 403
x86_64-darwin llvm-16.0.6 333
x86_64-darwin gtk+-2.24.33 277
x86_64-darwin mariadb-connector-odbc-3.1.14 225
x86_64-darwin postgresql-14.9 214
x86_64-darwin libmodplug-0.8.9.0 169
x86_64-darwin libdv-1.0.0 159
x86_64-darwin lynx-2.8.9rel.1 129
x86_64-darwin shared-mime-info-2.2 124
x86_64-darwin geos-3.11.2 107
x86_64-darwin postgresql-13.12 107
x86_64-darwin postgresql-15.4 107
x86_64-darwin postgresql-16.0 107
x86_64-darwin postgresql-12.16 104
x86_64-darwin postgresql-11.21 97
x86_64-darwin guile-3.0.9 91
x86_64-darwin source-highlight-3.1.9 73
x86_64-darwin gperf-3.0.4 72
x86_64-darwin usrsctp-0.9.5.0 65
x86_64-darwin v8-9.7.106.18 64
x86_64-darwin qtmultimedia-5.15.10 63
x86_64-darwin jemalloc-5.3.0 52
x86_64-darwin mecab-nodic-0.996 51
x86_64-darwin libiscsi-1.19.0 50
x86_64-darwin llvm-10.0.1 50
x86_64-darwin R-4.3.1 48
x86_64-darwin scdoc-1.11.2 40
x86_64-darwin libsecret-0.20.5 38
x86_64-darwin uw-imap-2007f 36
x86_64-linux compiler-rt-static-wasm32-unknown-wasi-16.0.6 34
x86_64-linux gnustep-base-1.28.0 34
x86_64-darwin perl5.38.0-XML-LibXML-2.0208 30
x86_64-darwin crystal-1.9.2 29
x86_64-darwin expect-5.45.4 27
x86_64-darwin gf2x-1.3.0 26
x86_64-darwin python-2.7.18.6 26
x86_64-darwin chicken-4.13.0 26
x86_64-linux pyside2-5.15.10 26
x86_64-darwin enchant-2.5.0 26
x86_64-darwin libgee-0.20.6 25
x86_64-darwin yaml-cpp-0.7.0 20
x86_64-darwin perl5.36.1-XML-LibXML-2.0208 19
x86_64-darwin source 18
x86_64-darwin python3.10-uamqp-1.6.4 18
x86_64-darwin python3.11-uamqp-1.6.4 18
x86_64-darwin vtk-9.2.6 18
x86_64-darwin inetutils-2.4 17
x86_64-darwin fmt-0.6.3.0 16
x86_64-darwin python3.11-pyerfa-2.0.0.1 16
x86_64-darwin subversion-1.14.2 15
x86_64-darwin ffmpeg-4.4.4 15
x86_64-darwin jxrlib-1.1 15
x86_64-darwin audiofile-0.3.6 15
x86_64-darwin libsmi-0.5.0 13
x86_64-darwin perl5.38.0-Curses-1.37 12
x86_64-darwin aeson-1.5.6.0 12
x86_64-darwin mbedtls-3.4.1 12
x86_64-darwin netsurf-libparserutils-0.2.4 12
x86_64-darwin netpbm-11.3.4 11
x86_64-darwin aiger-1.9.9 11
x86_64-linux algol68g-3.3.22.tar.gz 10
x86_64-darwin nodejs-20.6.1 10
x86_64-darwin gengetopt-2.23 10
x86_64-darwin rlwrap-0.46.1 10
x86_64-darwin gecode-3.7.3 10
x86_64-darwin inline-c-cpp-0.5.0.1 9
x86_64-darwin ppl-1.2 9
x86_64-darwin ocaml4.14.1-bap-2.5.0 9
x86_64-darwin sfml-2.5.1 8
x86_64-darwin diskdev_cmds-593 8
x86_64-darwin texlive-dvisvgm.bin-2022 8
x86_64-darwin coinutils-2.11.9 8
x86_64-darwin brick-0.70.1 8
x86_64-darwin libb64-2.0.0.1 8
x86_64-darwin wolfssl-5.5.4 8
x86_64-linux omcompiler-1.18.0 8
x86_64-darwin inkscape-1.2.2 7
x86_64-darwin python3.10-pyerfa-2.0.0.1 7
x86_64-darwin glm-0.9.9.8 6
x86_64-darwin chicken-sdl2-0.4.1 6
x86_64-darwin chicken-socket-0.3.3 6
x86_64-darwin dleyna-core-0.7.0 6
x86_64-darwin perl5.38.0-PathTools-3.75 6
x86_64-darwin iverilog-12.0 6
x86_64-darwin python3.11-python-axolotl-curve25519-0.4.1.post2 6
x86_64-darwin cogl-1.22.8 5
x86_64-darwin aalib-1.4rc5 5
x86_64-darwin libicns-0.8.1 5
x86_64-darwin cvs-1.12.13 5
x86_64-darwin python3.10-m2crypto-0.39.0 5
x86_64-darwin ruby3.1.4-ffi-1.10.0 5
x86_64-darwin libAfterImage-1.20 5
x86_64-darwin augeas-1.12.0 5
x86_64-darwin perl5.36.1-IPC-ShareLite-0.17 5
x86_64-darwin coordgenlibs-3.0.2 5
x86_64-linux python3.10-onnxconverter-common-1.13.0 4
x86_64-linux python3.11-onnxconverter-common-1.13.0 4
x86_64-darwin exinst-base-0.9 4
x86_64-darwin plotutils-2.6 4
x86_64-darwin superlu-5.2.1 4
x86_64-darwin text_cmds-99 4
x86_64-darwin chicken-epoxy-0.2.2 4
x86_64-darwin stfl-0.24 4
x86_64-darwin tie-2.4 4
x86_64-linux deliantra-server-3.1 4
x86_64-darwin tecla-1.6.3 4
x86_64-darwin discount-2.2.7b 4
x86_64-darwin gcc-7.5.0 4
x86_64-darwin bup-0.33.2 4
x86_64-darwin spatial-math-0.2.7.0 4
x86_64-darwin foma-0.10.0alpha 4
x86_64-darwin telepathy-glib-0.24.2 4
x86_64-darwin lrzsz-0.12.20 4
x86_64-darwin microsoft-gsl-4.0.0 4
x86_64-darwin python3.10-ndtypes-unstable-2019-08-01 4
x86_64-darwin liberasurecode-1.6.3 4
x86_64-darwin readstat-1.1.9 4
x86_64-darwin python3.11-m2crypto-0.39.0 4
x86_64-darwin mcpp-2.7.2.1 4
x86_64-darwin lbfgs-0.1 3
x86_64-darwin encfs-1.9.5 3
x86_64-darwin unar-1.10.7 3
x86_64-darwin motif-2.3.8 3
x86_64-darwin libxml++-2.40.1 3
x86_64-darwin gcc-10.5.0 3
x86_64-darwin smpeg-0.4.5 3
x86_64-darwin gcc-9.5.0 3
x86_64-darwin ruby3.1.4-sqlite3-1.4.2 3
x86_64-darwin hdf-4.2.15 3
x86_64-darwin llvm-11.1.0 3
x86_64-darwin ruby3.1.4-nokogiri-1.10.3 3
x86_64-darwin moarvm-2023.08 3
x86_64-darwin perl5.36.1-Curses-1.37 3
x86_64-darwin perl5.38.0-DBM-Deep-2.0016 3
x86_64-darwin perl5.38.0-CSS-DOM-0.17 3
x86_64-darwin python3.10-python-axolotl-curve25519-0.4.1.post2 3
x86_64-darwin catdvi-0.14 3
x86_64-darwin python3.11-livestreamer-1.12.2 3
x86_64-darwin ruby3.1.4-nokogiri-1.10.1 3
x86_64-darwin python3.10-packaging-21.3 3
x86_64-linux gnat-11.4.0 2
x86_64-linux liblinphone-5.2.98 2
x86_64-linux python3.11-msprime-1.2.0 2
x86_64-linux qt-libs-20210531-git 2
x86_64-linux python3.10-pillow-9.5.0 2
x86_64-darwin dictd-1.13.1 2
x86_64-darwin skews-0.1.0.3 2
x86_64-linux python3.10-unstructured-inference-0.5.22 2
x86_64-darwin NSPlist-unstable-2017-04-11 2
x86_64-darwin aria2-1.36.0 2
x86_64-darwin rocksdb-6.23.3 2
x86_64-linux ocaml-bs-4.06.1 2
x86_64-linux CCEmuX-cct.jar 2
x86_64-darwin vorbis-tools-1.4.2 2
x86_64-darwin cctools-973.0.1 2
x86_64-darwin xqilla-2.3.4 2
x86_64-linux btrfs-progs-6.5.1 2
x86_64-linux deadbeef-1.9.5 2
x86_64-darwin wordnet-3.0 2
x86_64-darwin cdrtools-3.02a09 2
x86_64-linux perl5.38.0-SDL-2.548 2
x86_64-darwin gforth-0.7.9_20230518 2
x86_64-darwin gcc-6.5.0 2
x86_64-darwin gcc-8.5.0 2
x86_64-darwin ruby3.1.4-nio4r-2.5.8 2
x86_64-linux petsc-3.19.2 2
x86_64-linux pangomm-2.42.2 2
x86_64-linux python3.10-git-annex-adapter-0.2.2 2
x86_64-darwin tetex-3.0 2
x86_64-darwin Xaw3d-1.6.3 2
x86_64-darwin sym-0.13.0 2
x86_64-darwin HsSyck-0.53 2
x86_64-darwin EBTKS-unstable-2017-09-23 2
x86_64-darwin kakoune-unwrapped-2023.08.05 2
x86_64-darwin perl5.38.0-Mac-Pasteboard-0.011 2
x86_64-darwin coredns-1.11.0 2
x86_64-darwin the-foundation-1.6.1 2
x86_64-darwin llvm-7.1.0 2
x86_64-darwin lua5.1-mpack-1.0.9-0 2
x86_64-darwin lua5.2-mpack-1.0.9-0 2
x86_64-darwin lua5.4-mpack-1.0.9-0 2
x86_64-darwin liblxi-1.20 2
x86_64-darwin libmcrypt-2.5.8 2
x86_64-linux ruby3.1.4-glib2-3.4.1 2
x86_64-darwin python3.10-aioopenssl-0.6.0 2
x86_64-darwin trellis-unstable-2022-09-14 2
x86_64-darwin dosbox-0.74-3 2
x86_64-linux or-tools-9.4 2
x86_64-darwin premake5-5.0.0-beta2 2
x86_64-linux python3.10-typeguard-2.13.3 2
x86_64-darwin whereami-unstable-2022-02-18 2
x86_64-darwin gfan-0.6.2 2
x86_64-linux python3.10-pyvo-1.4 2
x86_64-linux python3.10-pomegranate-0.14.8 2
x86_64-darwin python3.10-deal-solver-0.1.1 2
x86_64-linux python3.10-paramz-0.9.5 2
x86_64-darwin python3.10-netCDF4-1.6.2 2
x86_64-darwin python3.10-para-0.0.8 2
x86_64-darwin vectorscan-5.4.10.1 2
x86_64-linux python3.11-pyvo-1.4 2
x86_64-linux python3.11-pomegranate-0.14.8 2
x86_64-linux python3.11-paramz-0.9.5 2
x86_64-darwin python3.11-netCDF4-1.6.2 2
x86_64-darwin python3.11-para-0.0.8 2
x86_64-linux python3.11-inkex-1.2.2 2
x86_64-linux rippled-1.9.4 2
x86_64-darwin ruby3.1.4-sqlite3-1.4.4 2
x86_64-darwin scheme48-1.9.2 2
x86_64-darwin tcl-8.5.19 2
x86_64-darwin tbb-2021.8.0 2
x86_64-darwin pe-parse-2.0.0 2
x86_64-darwin age-1.1.1 1
x86_64-darwin python3.10-proxmoxer-2.0.1 1
x86_64-darwin pv-1.7.24 1
x86_64-darwin mighty-metropolis-2.0.0 1
x86_64-darwin rose-trees-0.0.4.5 1
x86_64-darwin ruby3.1.4-lmdb-0.5.3 1
x86_64-darwin ruby3.1.4-mathematical-1.6.14 1
x86_64-darwin monero-cli-0.18.2.2 1
x86_64-darwin bazel-5.4.1 1
x86_64-darwin bird-lg-frontend-1.3.1 1
x86_64-darwin spooles-2.2 1
x86_64-darwin chicken-scheme2c-compatibility-1.8 1
x86_64-darwin gamin-0.1.10 1
x86_64-darwin perl5.38.0-IO-Pty-1.16 1
x86_64-darwin fpc-3.2.2 1
x86_64-darwin python3.10-watchfiles-0.20.0 1
x86_64-darwin dovecot-2.3.21 1
x86_64-darwin opam-2.1.5 1
x86_64-darwin hevm-0.51.3 1
x86_64-darwin base16-bytestring-1.0.2.0 1
x86_64-darwin Glob-0.10.2 1
x86_64-darwin Diff-0.4.1 1
x86_64-darwin tokei-12.1.2 1
x86_64-darwin texinfo-4.13a 1
x86_64-darwin libshout-2.4.6 1
x86_64-darwin ruby3.1.4-cool.io-1.7.1 1
x86_64-darwin rocksdb-7.10.2 1
x86_64-darwin gcc-4.8.5 1
x86_64-darwin jogl-2.4.0 1
x86_64-darwin gfortran-10.5.0 1
x86_64-darwin gfortran-4.8.5 1
x86_64-darwin gfortran-6.5.0 1
x86_64-darwin gfortran-7.5.0 1
x86_64-darwin gfortran-8.5.0 1
x86_64-darwin gfortran-9.5.0 1
x86_64-darwin ORBit2-2.14.19 1
x86_64-darwin pkcs11-helper-1.29.0 1
x86_64-darwin async-refresh-0.3.0.0 1
x86_64-darwin epub-metadata-5.2 1
x86_64-darwin QuadProgpp-4b6bd65f09fbff99c172a86d6e96ca74449b323f 1
x86_64-darwin swisstable-0.1.0.3 1
x86_64-darwin posix-timer-0.3.0.1 1
x86_64-darwin posix-socket-0.3 1
x86_64-darwin select-0.4.0.1 1
x86_64-darwin libserdes-6.2.0 1
x86_64-darwin scas-0.5.5 1
x86_64-darwin ocaml4.14.1-z3-4.8.17 1
x86_64-darwin ustream-ssl-unstable-2023-02-25 1
x86_64-darwin llvm-8.0.1 1
x86_64-darwin llvm-9.0.1 1
x86_64-darwin lua5.3-mpack-1.0.9-0 1
x86_64-darwin l-smash-2.14.5 1
x86_64-darwin cbc-2.10.4 1
x86_64-darwin gcab-1.6 1
x86_64-darwin antlr-2.7.7 1
x86_64-darwin texlive-pdfbook2.bin-1.4 1
x86_64-darwin texlive-mflua.bin-62774 1
x86_64-darwin python3.9-ruamel-yaml-clib-0.2.7 1
x86_64-darwin odin-dev-2023-08 1
x86_64-darwin dosbox.desktop 1
x86_64-darwin SPIRV-LLVM-Translator-unstable-2022-05-04 1
x86_64-darwin jam-2.6.1 1
x86_64-darwin perl5.36.1-BDB-1.92 1
x86_64-darwin perl5.36.1-Math-Pari-2.030518 1
x86_64-darwin perl5.38.0-BDB-1.92 1
x86_64-darwin perl5.38.0-Math-Pari-2.030518 1
x86_64-darwin varnish-7.2.1 1
x86_64-darwin python3.10-caio-0.9.13 1
x86_64-darwin python3.10-pyheck-0.1.5 1
x86_64-darwin python3.10-simpy-4.0.1 1
x86_64-darwin python3.11-caio-0.9.13 1
x86_64-darwin python3.11-aioopenssl-0.6.0 1
x86_64-darwin python3.11-pyheck-0.1.5 1
x86_64-darwin python3.11-simpy-4.0.1 1
x86_64-darwin python3.11-ndtypes-unstable-2019-08-01 1
x86_64-darwin avr-stage-final-gcc-8.5.0 1
x86_64-darwin avr-stage-static-gcc-12.3.0 1
x86_64-darwin xmlrpc-c-1.51.07 1
x86_64-darwin lp_solve-5.5.2.11 1
x86_64-darwin ruby3.1.4-gpgme-2.0.20 1
x86_64-darwin ruby3.1.4-ffi-1.9.25 1
x86_64-darwin ruby3.1.4-ffi-1.9.21 1
x86_64-darwin ruby3.1.4-oj-2.18.1 1
x86_64-darwin ruby3.1.4-thin-1.7.2 1
x86_64-darwin StormLib-9.22 1
x86_64-darwin ruby3.1.4-msgpack-1.2.6 1
x86_64-darwin python3.10-pymupdf-1.22.5 1
x86_64-darwin rman-3.2 1
x86_64-darwin torsocks-2.3.0 1
x86_64-linux nixpkgs-metrics 1
x86_64-darwin ruby3.1.4-grpc-1.53.0 1
x86_64-darwin ruby3.1.4-curses-1.4.4 1
x86_64-darwin proj-7.2.1 1
x86_64-darwin cxx-rs-1.0.94 1

Generated by eval-report

reckenrode commented 1 year ago

Thanks for the report. It’s heartening that the most severe offenders are fixed already on staging or have open PRs with fixes.

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-macos-monthly/12330/42

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-macos-monthly/12330/44

reckenrode commented 1 year ago

The LLVM update has been merged into master.