unknown NIX_CFLAGS_COMPILE flag causes compliation to fail #18998

Open nektro opened 8 months ago

Zig Version

0.11.0

Steps to Reproduce and Observed Behavior

https://git.sr.ht/~nektro/magnolia-desktop
HEAD is 972f719bbd3de730d6f19a5ef52175486db365b7 at time of writing
nix-shell
zig build -Dall

after upgrading from nixos 23.05 to 23.11

zig build-exe magnolia-demo-mouse Debug native: error: warning: Unrecognized C flag from NIX_CFLAGS_COMPILE: -idirafter
warning: Unrecognized C flag from NIX_CFLAGS_COMPILE: /usr/include

zig build-exe magnolia-demo-7guis-counter Debug native: error: warning: Unrecognized C flag from NIX_CFLAGS_COMPILE: -idirafter
warning: Unrecognized C flag from NIX_CFLAGS_COMPILE: /usr/include

zig build-exe magnolia-demo-menubar Debug native: error: warning: Unrecognized C flag from NIX_CFLAGS_COMPILE: -idirafter
warning: Unrecognized C flag from NIX_CFLAGS_COMPILE: /usr/include

....

Expected Behavior

ignore the ones it doesnt recognize, a successful compile

I don't think zig should try to interpret these env vars at all. The nix specific code in zig IMO is misguided.

as a workaround I made a custom 0.11 build with /lib/std/zig/system/NativePaths.zig#L42 deleted, which allowed me to continue work on my project

I did run into the same issue. I currently solve that issue by running unset in my development shells shellHook:

  shellHook = ''
    # We unset some NIX environment variables that might interfere with the zig
    # compiler.
    # Issue: https://github.com/ziglang/zig/issues/18998
    unset NIX_CFLAGS_COMPILE
    unset NIX_LDFLAGS
  '';

@urso using that i either get error: error: unable to find Dynamic system library 'X11' using strategy 'paths_first'. or error: 'X11/Xlib.h' file not found depending on which flag i tell it to unset

That means your flake / nix env does not have correct buildInputs or you are not including pkg-config as nativeBuildInputs

don't think so because it works without the shellHook (when i was on 23.05)

That's because zig is being too smart and reads those env vars, aka it's doing the bad thing and hiding build bugs in nix sandbox. I suggest including pkg-config in your nativeBuildInputs, that's the correct way of dealing with system deps.

good catch, so this is the diff that fixed my build

diff --git a/shell.nix b/shell.nix
index 0f9e8d5..e25da0c 100644
--- a/shell.nix
+++ b/shell.nix
@@ -5,7 +5,14 @@ pkgs.mkShell {
     gcc
     xorg.libX11
     mesa_glu
+    pkg-config
   ];

   hardeningDisable = [ "all" ];
+
+  # https://github.com/ziglang/zig/issues/18998
+  shellHook = ''
+    unset NIX_CFLAGS_COMPILE
+    unset NIX_LDFLAGS
+  '';
 }

thanks both

update: the output binaries are not runnable

[nix-shell:/run/media/meghan/dev/magnolia-desktop]$ ldd ./zig-out/bin/magnolia-Calculator
        linux-vdso.so.1 (0x00007ffea2bbf000)
        libX11.so.6 => not found
        libGL.so.1 => not found
        libc.so.6 => /lib/libc.so.6 (0x00007f28303f9000)
        /nix/store/xmprbk52mlcdsljz66m8yf7cf0xf36n1-glibc-2.38-44/lib/ld-linux-x86-64.so.2 => /nix/store/xmprbk52mlcdsljz66m8yf7cf0xf36n1-glibc-2.38-44/lib64/ld-linux-x86-64.so.2 (0x00007f28305e7000)

That's normal (on nixos), you need to run them in the develop shell that sets LD_LIBRARY_PATH, make a nix package that uses autoPatchelfHook, or use something like steam-run. It's also the reason zig2nix has convenience shells for this: https://github.com/Cloudef/zig2nix?tab=readme-ov-file#convenience-shell-for-multimedia-programs as well as ability to put any custom runtime deps in the zig-env if you use zig2nix flake: https://github.com/Cloudef/zig2nix/blob/master/flake.nix#L70

do you know how? https://nixos.wiki/wiki/Packaging/Binaries#Using_AutoPatchelfHook seems incompatible with my workflow

If you don't want to package, on nixos your options are set LD_LIBRARY_PATH, use steam-run or something like nix-ld https://github.com/Mic92/nix-ld

(Or use zig2nix and you get packaging for free as well)

imo this is a regression, im using nix-shell to expose the pkg-config and the library paths during build. zig should embed them or provide an option for me to instruct it to

No zig should not do any smart decisions here. The problem is that you are linking against soname library, and this is the correct linking behavior with soname libraries, it does not link to the absolute path of that library but rather the "soname". The fact that the libraries are not found by default is nixos specific issue and you either have to use different distro or do it the nixos way. If you want to still embed the paths, you can instruct zig to insert multiple rpaths to the linked executable.

The NativePaths.zig parses NIX_LDFLAGS to set rpaths https://github.com/ziglang/zig/blob/0.11.0/lib/std/zig/system/NativePaths.zig#L50-L55 thats why it worked for you before, but it's again something zig really should not do as these env variables are not meant to be used outside of nixpkgs.

If zig wants to rely on env vars it should use the standard ones:

LDFLAGS
CFLAGS
CXXFLAGS
....

And to get the current zig behaviour one can do LDFLAGS=$NIX_LDFLAGS inside a nix shell.

ziglang / zig