Use of $NIX_BUILD_TOP considered harmful

[bjornfor]

Describe the bug

Nix expressions that rely on $NIX_BUILD_TOP may break under nix-shell (or when using nativeBuildInputs = [ keepBuildTree ]), so let's try to avoid it.

Steps To Reproduce

Steps to reproduce the behavior:

1. Find a package that uses $NIX_BUILD_TOP, e.g. spotifyd via cargo setup hook: nix-shell -A spotifyd
2. Run genericBuild inside the shell.
3. Watch it fail trying to work with files inside $NIX_BUILD_TOP (when the files actually are in $PWD).

Expected behavior

Builds work both with nix-build and nix-shell (not fail in the latter case).

Additional context

A workaround for the nix-shell breakage could be exporting $NIX_BUILD_TOP before invoking any build code. (nix develop has code for that?) But it wouldn't fix the keepBuildTree case.

Notify maintainers

TODO list

• [x] cargo setup hook: https://github.com/NixOS/nixpkgs/issues/138554
• [ ] buildBazelPackage
• [ ] Most other packages referring to $NIX_BUILD_TOP. (Some are harmless, need to check them.)

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/use-of-nix-build-top-considered-harmful/21443/1

[bjornfor]

CC bazel and cargo setup hook people: @abbradar @danieldk.

Do you agree with the issue? Who else need to be aware of this issue before we start removing $NIX_BUILD_TOP from nixpkgs?

[roberth]

I think we should be able to debug builds in the real sandbox or a good approximation of it. We should accept that nix-shell has ossified and needs a replacement not just for its role as "dev shell" but also for its role as build debugging shell.

Nonetheless, it's good for Nixpkgs to try to be compatible with usage in dev shells.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/temporary-writable-directory-for-builds/34907/1

[bjornfor]

Another fix here: https://github.com/NixOS/nixpkgs/pull/266723

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-look-at-the-build-directory-after-successful-build/14007/3