numtide / devshell

Per project developer environments
https://numtide.github.io/devshell/
MIT License
1.22k stars 87 forks source link

fix(extra/rust): xcbuild requires 2 paths to be exposed by env #266

Closed Pegasust closed 1 year ago

Pegasust commented 1 year ago

Symptoms

```console $ cargo build error: linking with `cc` failed: exit status: 134 | = note: LC_ALL="C" PATH="/nix... ... = note: xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun libc++abi: terminating with uncaught exception of type std::runtime_error: codesign_failed: 1 /nix/store/pz87iqygyl6gklpqlwd8bjs8sqd6hvl3-post-link-sign-hook: line 4: 74910 Abort trap: 6 CODESIGN_ALLOCATE=codesign_allocate /nix/store/ss3qicb0gzh00f87c6xs9nfdj92mad8k-sigtool-0.1.3/bin/codesign -f -s - "$linkerOutput" ``` This is actually a mint cargo workspace with `lsp`, and many other "hello world" crates ```console $ cargo build error: linking with `cc` failed: exit status: 134 | = note: LC_ALL="C" PATH="/nix/store/40rb2myhi22n2jr3a1f5r7n673j1llcw-rust-minimal-1.72.0-nightly-2023-06-29/lib/rustlib/aarch64-apple-darwin/bin:/nix/store/hayj70xixmzri25f0c2vbdrn1q3xy31s-devs hell-dir/bin:/nix/store/ysnid4p0w4r9kg9834i327j4lxhcb50f-bash-interactive-5.2-p15/bin:/path-not-set:/Users/hungtran/local_repos/zork/.bin:/Users/hungtran/.nix-profile/bin:/nix/var/nix/profiles/de fault/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/hungtran/Applications/Home Manager Apps/kitty.app/Contents/MacOS:/nix/store/cp70ajpjw8arik7x8rrq27w1swv 22s9r-neovim-ruby-env/bin:/nix/store/c6jz0w2gwc4clmhawsmh203yrh31pq6m-nodejs-18.16.1/bin:/nix/store/ihn63vylyqmjjgr63r9shzvrjbfv16jg-tree-sitter-0.20.8/bin:/nix/store/vbn3abzrfj32qy59g868hb7ndrwk igl0-fzf-0.42.0/bin:/nix/store/ndy7daf60jsqwaqzm5b8g44fvra43sc0-ripgrep-13.0.0/bin:/nix/store/dkjx87xjnh7kk2yjbsqw608jgpp2a4j6-zk-0.14.0/bin:/nix/store/s55k81yblx0k1qdily40n7jl67215ivs-fd-8.7.0/b in:/nix/store/4ypgz0lrii62saqgrcn432sa13clqns0-clang-11.1.0-lib/bin:/nix/store/pi2c2p233k099y0d21mwjkhnyg8r9wa5-nil-2023-05-09/bin:/nix/store/bx4g81bdsq9mlby0dijlr82xcgwk8mry-rust-minimal-1.72.0- nightly-2023-06-29/bin:/nix/store/wkrvjivxffs6g7q728n12mgb427yhggj-nickel-1.0.0/bin:/nix/store/vjb7cj5ljmkqljcnzj6rmbc2n24vh9c4-nls-1.0.0/bin:/nix/store/fjbcw6iz6h3gyi6yigin682gcsbn6994-sg.nvim/b in:/nix/store/1qmphwp6j96aimch0z9rjq7xlrkf3dyv-go-1.20.5/bin:/nix/store/vxs6h5zj2lnij97w207ml2bqv4bmqqya-lua-language-server-3.6.22/bin:/nix/store/i2ga2q14vr1dwbh3iz6nk06jqi451gz8-pyright-1.1.316 /bin:/nix/store/cfmx8v5iigf3i0rm1z2wdyfn2bqpy9hp-python3.10-pylint-2.16.2/bin:/nix/store/dhj27qx5c2yx4bzq12j18yzlsvyqqbg0-python3.10-flake8-6.0.0/bin:/nix/store/9prmj5wrivli1ijshg09gd8j5zzq07iv-a pple-framework-System-11.0.0/bin:/nix/store/vfvlqvyzyvhns7gyz1bvgj8q6307i4rm-apple-framework-CoreFoundation-11.0.0/bin:/Users/hungtran/.local/share/nvim/mason/bin" VSLANG="1033" ZERO_AR_DATE="1" "cc" "-arch" "arm64" "/tmp/rustcMg0rTh/symbols.o" "/Users/hungtran/local_repos/zork/target/debug/deps/lsp-d92861ee639d84ad.1jr4o2kw6y3ksl0u.rcgu.o" "/Users/hungtran/local_repos/zork/target/debug/ deps/lsp-d92861ee639d84ad.204ae4tecutcekgy.rcgu.o" "/Users/hungtran/local_repos/zork/target/debug/deps/lsp-d92861ee639d84ad.3moztt19cmw0o1ks.rcgu.o" "/Users/hungtran/local_repos/zork/target/debug /deps/lsp-d92861ee639d84ad.46lr2edtiuegj2kp.rcgu.o" "/Users/hungtran/local_repos/zork/target/debug/deps/lsp-d92861ee639d84ad.50vptv1ge4mtdwgr.rcgu.o" "/Users/hungtran/local_repos/zork/target/debu g/deps/lsp-d92861ee639d84ad.frtdyazq0exhm9p.rcgu.o" "/Users/hungtran/local_repos/zork/target/debug/deps/lsp-d92861ee639d84ad.31rjo0bzdr75oec0.rcgu.o" "-L" "/Users/hungtran/local_repos/zork/target /debug/deps" "-F" "/nix/store/hayj70xixmzri25f0c2vbdrn1q3xy31s-devshell-dir/Library/Frameworks" "-L" "/nix/store/40rb2myhi22n2jr3a1f5r7n673j1llcw-rust-minimal-1.72.0-nightly-2023-06-29/lib/rustli b/aarch64-apple-darwin/lib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-05943b50472cd2d6.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-d4df49eb937c25fb.rlib" "/nix/store/57phj a1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-d9455aebba8055d5.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26j xga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libmemchr-218138b1ab541b3e.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nigh tly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-d187e6473663e7f6.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch6 4-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-2b31aea7ba1669cf.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustli b/aarch64-apple-darwin/lib/librustc_demangle-c1dfb86e23640844.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple- darwin/lib/libstd_detect-27914095f0dba818.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbr own-b8e271a3426f8722.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-4 96f42c51ffe7099.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libminiz_oxide-6bd89cd5d42afec5.rli b" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libadler-37f6ce1bec5f08ec.rlib" "/nix/store/57phja1hxk 5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-6d20841979b6d418.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-r ust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-ecea5914c2ef473a.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2 023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-cc741fbf27f447be.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-da rwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-4e0b06e5749dd743.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64- apple-darwin/lib/librustc_std_workspace_core-70a0087d6881bc8d.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple- darwin/lib/libcore-6fc8bef838a5948d.rlib" "/nix/store/57phja1hxk5wlz49ia6gp3k98n26jxga-rust-std-1.72.0-nightly-2023-06-29-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_bui ltins-a2bfdcafe5010b58.rlib" "-lSystem" "-lc" "-lm" "-L" "/nix/store/40rb2myhi22n2jr3a1f5r7n673j1llcw-rust-minimal-1.72.0-nightly-2023-06-29/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/hun gtran/local_repos/zork/target/debug/deps/lsp-d92861ee639d84ad" "-Wl,-dead_strip" "-nodefaultlibs" = note: xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun libc++abi: terminating with uncaught exception of type std::runtime_error: codesign_failed: 1 /nix/store/pz87iqygyl6gklpqlwd8bjs8sqd6hvl3-post-link-sign-hook: line 4: 74910 Abort trap: 6 CODESIGN_ALLOCATE=codesign_allocate /nix/store/ss3qicb0gzh00f87c6xs9nfdj92mad8k-si gtool-0.1.3/bin/codesign -f -s - "$linkerOutput" clang-11: error: linker command failed with exit code 134 (use -v to see invocation) error: could not compile `lsp` (bin "lsp") due to previous error ```

The fix explained

xcbuild on its own require 2 entries of $PATH to be appended so that rustc, thus cargo build, can see and use xcrun via $PATH

Potential upstream fixes and alternative approaches

https://github.com/NixOS/nixpkgs/pull/242431

But the PR will still likely not resolve this in an elegant way. The way xcbuild is structured, collocating xcbuild and its xcrun via symlinkJoin will fail the build due to sharing (iirc) /share symlinks.

How the traditional route of pkgs.mkShell works was via a hack of adding a string-path onto propagatedBuildInputs so that it gets added to environment variable. devshell despises this approach in favor for minimal env via mkNakedShell.

Alternative approaches

Why an alternative approach? The current fix wishes for pkgs.xcbuild to coincide with the one rustc uses. With the popularity of many rust-overlays, this is very rarely the case. A drift of version is fine (for now), but it will require the end user to fetch xcbuild twice, and may break in dep requirements (unlikely since xcbuild is archived. The following approaches take these potential pitfalls into account.

a. I'm not confident enough for an attempt of refactoring xcbuild so that it is friendly for collocation with xcrun. From the back of my mind, this will likely drastically change many dependent derivations. Even xcbuild.withXcrun would be a better solution that solves at the upstream level.

b. Wiggle mkNakedShell, which I'm also not too confident in, to take into account of propagatedBuildInputs and other pre-req derivations.

c. A more bold approach is to just disregard xcbuild entirely and use xcrun on our Rust environment. At the time of discovery, I didn't note this down in more details, so take this with a grain of salt.

Pegasust commented 1 year ago

Hi, any interest in merging this? Is there anyway I could contribute to help getting this merged?

zimbatm commented 1 year ago

yup, thank you. I just got overloaded with notifications.