Open erikarvstedt opened 5 months ago
why are you saying those lib directories are include paths? What files are in there?
Ah, sorry, these *-dev
paths are not only present in NIX_CFLAGS_COMPILE
, but also in other env vars.
Let's take /nix/store/nhv7xmxzi63lp3x1j7zrwb7f2248sgl2-libxml2-2.11.5-dev as an example.
It's included in these env vars: |
Env var | Env var entry |
---|---|---|
CMAKE_INCLUDE_PATH | /nix/store/nhv7xmxzi63lp3x1j7zrwb7f2248sgl2-libxml2-2.11.5-dev/include |
|
CMAKE_PREFIX_PATH | /nix/store/nhv7xmxzi63lp3x1j7zrwb7f2248sgl2-libxml2-2.11.5-dev |
|
NIX_CFLAGS_COMPILE | -isystem /nix/store/nhv7xmxzi63lp3x1j7zrwb7f2248sgl2-libxml2-2.11.5-dev/include |
|
PATH | /nix/store/nhv7xmxzi63lp3x1j7zrwb7f2248sgl2-libxml2-2.11.5-dev/bin |
Contents:
/nix/store/nhv7xmxzi63lp3x1j7zrwb7f2248sgl2-libxml2-2.11.5-dev
├── bin
│ └── xml2-config
├── include
│ └── libxml2
│ └── libxml
│ ├── c14n.h
│ ├── ...
├── lib
│ ├── cmake
│ │ └── libxml2
│ │ └── libxml2-config.cmake
│ └── pkgconfig
│ └── libxml-2.0.pc
├── nix-support
│ └── propagated-build-inputs
└── share
└── aclocal
└── libxml.m4
The same goes for all other *-dev
paths. They are dev
pkg outputs that contain build-related tooling and source files but no binary objects.
Hmm. When I do the following on my system:
$ nix-shell -p libxml2
$ env | grep NIX_LDFLAGS
NIX_LDFLAGS=-rpath /nix/store/2swpy93b6y5bsvmb7jawlmwwipwxmy2m-shell/lib64 -rpath /nix/store/2swpy93b6y5bsvmb7jawlmwwipwxmy2m-shell/lib -L/nix/store/8mw6ssjspf8k1ija88cfldmxlbarl1bb-zlib-1.2.13/lib -L/nix/store/awj9x4rwndilh9hcwf121yz3xr9j64w6-libxml2-2.10.4/lib -L/nix/store/8mw6ssjspf8k1ija88cfldmxlbarl1bb-zlib-1.2.13/lib -L/nix/store/awj9x4rwndilh9hcwf121yz3xr9j64w6-libxml2-2.10.4/lib
$ ls /nix/store/awj9x4rwndilh9hcwf121yz3xr9j64w6-libxml2-2.10.4/lib
libxml2.la libxml2.so libxml2.so.2 libxml2.so.2.10.4
It looks to me like adding an rpath for that -L directory was the right move. I think the only confirmed regression in this issue is the missing directory that you identified above - the gcc one. I'm not convinced these other differences are problematic.
Yes, these are lib outputs in NIX_LDFLAGS
from pkg libxml2.lib
. They should be added to the rpath. #17917 didn't change that.
What changed is that libxml2.dev
outputs, which are not included in NIX_LDFLAGS
(as shown in the env var table above), are also added to the rpath.
NIX_LDFLAGS: -L/nix/store/8g3myidcxw02mv9pzyvwq9cv0y9nayi7-libxml2-2.11.5/lib (should be added to rpath)
NIX_CFLAGS_COMPILE (and others): /nix/store/nhv7xmxzi63lp3x1j7zrwb7f2248sgl2-libxml2-2.11.5-dev (should not be added to rpath)
Hmm, I see. The new implementation of -feach-lib-rpath
naively adds each library search directory to the rpath list regardless of whether a dynamic library was used from inside that directory. That should be adjusted.
Alternately, we could proceed with removing that feature entirely, as I mentioned in that PR, since the original motivation for -feach-lib-rpath
was NixOS anyway. On the other hand, -feach-lib-rpath
as opposed to adding an rpath for each -L
from NIX_LDFLAGS
creates a binary with a slightly more efficient runtime startup, because it prevents the dynamic linker from searching unnecessary directories to find dynamically linked libraries. With the current strategy of adding an RPath for each -L
argument in NIX_LDFLAGS
it could potentially add unnecessary RPaths if those are in the nix environment but not ultimately used by the executable.
What remains to be diagnosed is
/nix/store/dghjv6hfz0s0z4kffa5ahyw2mhp79215-gcc-12.3.0-lib/lib (Missing after #17917)
Only adding required rpath entries would be perfect.
nixpkgs has a stdlib feature (patchelf
, autoPatchelfHook
) that prunes unneeded rpaths, but this is inconvenient when using Zig as a standalone build tool, outside of Nix derivations.
The libstdc++.so missing from rpath issue is likely related to #18742
Zig somehow auto-detects libstdc++
via cmake.
Ideally, Zig should remove this heuristic or correctly setup the rpath on Nix, so that a successful build guarantees that the binary can load its libs correctly.
Zig somehow auto-detects
libstdc++
via cmake.
I believe it does this in build.zig
https://github.com/ziglang/zig/blob/d0c06ca7127110a8afeb0ef524a197049892db21/build.zig#L715
(There are also other non linux parts that directly link libstdc++)
Solutions include:
use-zig-libcxx
to use the included libc++FYI I complained about this in a other issue, but IMO zig should not use the nixpkgs specific env vars at all https://github.com/ziglang/zig/issues/18998#issuecomment-1975056100
is it possible to also add the path to libstdc++.so
to rpath?
https://github.com/ziglang/zig/blob/fb88cfdf6aa3fabba700d8340f025e4a3e0d3fb2/build.zig#L819
it's a bit hacky, but maybe something like
exe.addRPath(.{ .cwd_relative = path_unpadded[0..path_unpadded.len - 1 - objname.len] });
so that we can ensure the path to libstdc++.so
would ended up in rpath
Since #17917, when building Zig within a Nix environment, the rpath of the resulting Zig binary is faulty/broken:
libgcc
, the lib providing the direct dependencylibstdc++.so
, is missing from rpath. This causes an error when runningzig
:Previously,
libgcc
was correctly included in rpath.Include paths from
NIX_CFLAGS_COMPILE
are present in rpath. These paths contain no shared objects and should not be present in rpath. Examples:Details
Zig rpath before #17917:
Zig rpath after #17917:
When the Zig src is located in
/home/user/src/zig
, the path to the nonexisting dir/home/user/src/outputs/out/lib
is added to rpath. This is probably also an error, but this was already present before #17917.Reproduce
cc @Jan200101, @kubkon, @andrewrk