Emulated cross-architecture build leaves a broken rust-std derivation on NixOS

Describe the bug

Hello everyone,

I’ve been trying to configure a Rust project using Nix to produce binaries for Darwin (arm64, x86_64) and Linux (arm64, x86_64) systems, so people can easily download them from GitHub releases. I’m trying to use Nix as the build process this time, to have a reproducible build locally as well as on GitHub Actions.

Instead of going through the cross-compilation route, with logic to use either ${system}.packages or ${system}.pkgsCross.${targetSystem}, I’m attempting to use an emulated cross-platform toolchain with qemu-user over binfmt_misc on the Kernel.

So far, I've been able to use the following Nix expression to build a barebone Rust file using Nix in all platforms, as long as the qemu-user-static emulation happens on a Non-NixOS Host.

System	Architecture	Target	Emulated by	Works?
Darwin	aarch64 (m1)	x86_64	Rosetta	Yes ✅
Linux (NixOS)	amd64 (proxmox/lxc)	aarch64	Debian host w/ qemu-user-static	Yes ✅
Linux (NixOS)	amd64 (proxmox/vm)	aarch64	NixOS host w/ qemu	No ❌
Linux (NixOS)	aarch64 (docker)	x86_64	Alpine host w/ qemu	Yes ✅
Linux (Ubuntu)	aarch64 (docker)	x86_64	Alpine host w/ qemu	Yes ✅
Linux (Ubuntu)	x86_64 (docker)	aarch64	Alpine host w/ qemu	Yes ✅

As long as the cross-platform emulation happens on a non-NixOS layer, such as the LXD host, the Docker host or on a Non-NixOS VM, the generated rust-std compilation is valid.

Build logs

``` error: aborting due to previous error error: builder for '/nix/store/7kz25f4zqfyrfixcc3fj804kbd9s6ap0-out.drv' failed with exit code 1; last 10 log lines: > Running phase: buildPhase > error: linking with `cc` failed: exit status: 1 > | > = note: LC_ALL="C" PATH="/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/bin:/nix/store/adms7m11kx2skc7bhi0z48mjlyqk74sp-rustc-wrapper-1.75.0/bin:/nix/store/abrrb6s259fsi5szq5w92ni70x7sp9ih-patchelf-0.15.0/bin:/nix/store/z062sjm2ppd5jdcisdgaxvl8l08dq774-gcc-wrapper-13.2.0/bin:/nix/store/yk0nw4wb0y7hy187p6s5i1cayzj1nbya-gcc-13.2.0/bin:/nix/store/pabzgfm37yk00mpnkh4akmr0nzg97kqq-glibc-2.38-44-bin/bin:/nix/store/hbh2gmp1idyjzw2wdh50idi8q01xm6yb-coreutils-9.4/bin:/nix/store/v5252q77fafyra57c8rzq2svlhawd14f-binutils-wrapper-2.40/bin:/nix/store/d5zbc1vk664z7azivyv38fsg6ia5w35w-binutils-2.40/bin:/nix/store/hbh2gmp1idyjzw2wdh50idi8q01xm6yb-coreutils-9.4/bin:/nix/store/9pfnndp3l9aj4a905r4k1m6iwcbl6vpj-findutils-4.9.0/bin:/nix/store/mrycscd5wfpp11g9bjvlllf6bia9m4kq-diffutils-3.10/bin:/nix/store/f3dmzlahl1avdbcbanfyyq8l8aycrzbn-gnused-4.9/bin:/nix/store/5fn8gw10q5nq3wqz84q29j9zd9wxba6k-gnugrep-3.11/bin:/nix/store/lq5z0piqczqamgy5h9s6lixj7plvls2c-gawk-5.2.2/bin:/nix/store/d6p4xfwg72kvwr9xx0ai3wn5g4znz11p-gnutar-1.35/bin:/nix/store/k0g5khwhv60vzwh7n8w47mm6kachb2yj-gzip-1.13/bin:/nix/store/y0817cpgflxrl42jjgdbz8bpxlarncmc-bzip2-1.0.8-bin/bin:/nix/store/dq0vnn5gnllnibc52x2l8vfshz1k4zvr-gnumake-4.4.1/bin:/nix/store/2icvzb9pj2ryvsy1m9jf4kj2mbkgbczj-bash-5.2p26/bin:/nix/store/sf5mmdwxiv8navi5a5brj969sdrkhx7i-patch-2.7.6/bin:/nix/store/cxyn9ww64idkxgv02685c6qwvyisvzan-xz-5.4.6-bin/bin:/nix/store/j96xh64m4x03fi9b633d6n87qq4v7g1d-file-5.45/bin" VSLANG="1033" "cc" "/build/rustcsAWxVf/symbols.o" "main.main.6d5145388994a7b7-cgu.0.rcgu.o" "main.4k7vvgzx11b68cao.rcgu.o" "-Wl,--as-needed" "-L" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd-436329434b63751b.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libpanic_unwind-645486d646ae78ee.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libobject-05eb0ca8862e71d6.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libmemchr-e29b4003c65f7892.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libaddr2line-75516828d20fd799.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libgimli-48e15cafb03bf5af.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_demangle-6e6e48662894ac3c.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd_detect-27e76fbf8405ed1a.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libhashbrown-fc59afcb036257bc.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-0ce5a40297c0b123.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libminiz_oxide-44673f3a7f799a8a.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libadler-cf9241d85da8aafa.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libunwind-729ec6f67ecd9d0c.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcfg_if-14289be8725ff0e3.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-1ba610f6ee9c1082.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/liballoc-7510187c76712943.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_core-197ec3f9e11b1939.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcore-3134799bb6fa5396.rlib" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcompiler_builtins-04c9643a755e1271.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/nix/store/y3bpndhxdb1gfagnd2316x0833737n78-rustc-1.75.0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-o" "main" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs" > = note: /build/rustcsAWxVf/symbols.o: file not recognized: file format not recognized > collect2: error: ld returned 1 exit status > > > error: aborting due to previous error > For full logs, run 'nix log /nix/store/7kz25f4zqfyrfixcc3fj804kbd9s6ap0-out.drv'.``` ```

Steps To Reproduce

nix-build

To reproduce the behaviour, you can use the following expression:

Ensure your NixOS has cross-platform emulation enabled

# configuration.nix
{
# In the case of a x86_64 NixOS host, enable aarch64 emulation
boot.binfmt.emulatedSystems = ["aarch64-linux"];
}

Ensure emulation is working as expected calling a cross-platform Nix package

$ nix-shell -p hello --argstr system aarch64-linux --command hello
Hello, world!

Build the following expression with the emulated system
```
nix-build default.nix --argstr system aarch64-linux
```

# default.nix
{system ? builtins.currentSystem}: let
  pkgs = import <nixpkgs> {
    inherit system;
  };
in
  pkgs.stdenv.mkDerivation {
    name = "out";

    unpackPhase = "true";

    nativeBuildInputs = [
      pkgs.rustc
    ];

    buildPhase = ''
      echo 'fn main() { }' > main.rs
      rustc main.rs
    '';

    installPhase = ''
      cp main $out
    '';
  }

nix-shell

It's also possible to use a nix-shell command to enter a troubleshooting environment, and inspect the resulting symbols.o. I'm not familiar enough with investigating why the .o is considered malformed (yet) tho.

Ensure your NixOS has cross-platform emulation enabled

# configuration.nix
{
# In the case of a x86_64 NixOS host, enable aarch64 emulation
boot.binfmt.emulatedSystems = ["aarch64-linux"];
}

Ensure emulation is working as expected calling a cross-platform Nix package

$ nix-shell -p hello --argstr system aarch64-linux --command hello
Hello, world!

Create a nix-shell with rustc and attempt to compile, while keeping the temporary files

$ nix-shell --argstr system aarch64-linux -p rustc -p stdenv -p file

(nix-shell) $ echo 'fn main() { }' > main.rs

(nix-shell) $ rustc main.rs -C save-temps
[error with symbols.o temp path]

(nix-shell) $ file /run/user/1000/rustcOX7qBA/symbols.o
/run/user/1000/rustcOX7qBA/symbols.o: ELF 64-bit LSB relocatable, ARM aarch64, version 1 (SYSV), not stripped

(nix-shell) $ nm /run/user/1000/rustcOX7qBA/symbols.o
nm: /run/user/1000/rustcOX7qBA/symbols.o: file format not recognized

(nix-shell) $ objdump -t /run/user/1000/rustcOX7qBA/symbols.o
objdump: /run/user/1000/rustcOX7qBA/symbols.o: file format not recognized

(nix-shell) $ ld /run/user/1000/rustcOX7qBA/symbols.o
/run/user/1000/rustcOX7qBA/symbols.o: file not recognized: file format not recognized

Expected behavior

The compilation of the barebone Rust program should produce a cross-platform binary on result as using Nix on Non-NixOS platforms.

Additional context

I've created a repository with additional tests, including a flake-based build and a Github Actions task to showcase the expression works when running Nix on Ubuntu.

This is the minimal reproducible scenario I've been able to extract with just nixpkgs packages and non-experimental commands.

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.79, NixOS, 24.05 (Uakari), 24.05.20240224.9a9dae8`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Comparing the emulation binfmt_misc registration between a working host and NixOS, it seems there are a few missing flags.

Debian host

$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64
enabled
interpreter /usr/libexec/qemu-binfmt/aarch64-binfmt-P
flags: POCF
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00fffffffffffffffffeffffff

NixOS host

$ cat /proc/sys/fs/binfmt_misc/aarch64-linux
enabled
interpreter /run/binfmt/aarch64-linux
flags: P
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00ffffffffffff00fffeffffff

It seems that these flags are possible to be set on NixOS, but they are disabled by default.

To match exactly the same configuration as the Debian host, I've applied the following overrides:

{
  boot.binfmt.emulatedSystems = ["aarch64-linux"];
  boot.binfmt.registrations.aarch64-linux = {
    matchCredentials = true;
    fixBinary = true;
    mask = lib.mkForce ''\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'';
  };
}

This results in the same configuration options as the compared host:

$ cat /proc/sys/fs/binfmt_misc/aarch64-linux

enabled
interpreter /run/binfmt/aarch64-linux
flags: POCF
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00fffffffffffffffffeffffff

The interpreter points to a qemu-static-*-P binary, just like the Debian host.

$ file /run/binfmt/aarch64-linux
/run/binfmt/aarch64-linux: symbolic link to /nix/store/mp17zkcxaqmjyjbhb3pvr0ni2z8jgnzh-qemu-aarch64-binfmt-P-x86_64-unknown-linux-musl/bin/qemu-aarch64-binfmt-P

I've attempted a few combinations of parameters, such as wrapInterpreterInShell and preserveArgvZero without much success yet.

Even after a huge detour attempting to enable qemu-user-static, the results are still the same when running the emulated compilation tools under NixOS.

After applying the following patches:

And adjusting the /etc/nixos/configuration.nix:

boot = {
    binfmt = {
        emulatedSystems = ["aarch64-linux"];
        preferStaticEmulators = true;
        registrations = {
            aarch64-linux = {
                matchCredentials = true;
                mask = lib.mkForce ''\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'';
            };
        };
    };
};

I'm able to have a qemu-user-static running under binfmt_misc with the same flags as the Ubuntu host.

$ cat /proc/sys/fs/binfmt_misc/aarch64-linux

enabled
interpreter /run/binfmt/aarch64-linux
flags: POCF
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00fffffffffffffffffeffffff

$ file /run/binfmt/aarch64-linux

/run/binfmt/aarch64-linux: symbolic link to /nix/store/gapqjg8p33gmgj4hvdb300hpiggslyjr-qemu-static-x86_64-unknown-linux-musl-8.2.2/bin/qemu-aarch64

$ file  /nix/store/gapqjg8p33gmgj4hvdb300hpiggslyjr-qemu-static-x86_64-unknown-linux-musl-8.2.2/bin/qemu-aarch64

/nix/store/gapqjg8p33gmgj4hvdb300hpiggslyjr-qemu-static-x86_64-unknown-linux-musl-8.2.2/bin/qemu-aarch64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=0f92fecf54c877035eee6b8c29ce285b7ed03336, not stripped

$ nix-build default.nix --argstr system aarch64-linux

``` $ nix-build default.nix --argstr system aarch64-linux this derivation will be built: /nix/store/9nhm44h3jfj495gxzfh02514lna48k1q-out.drv building '/nix/store/9nhm44h3jfj495gxzfh02514lna48k1q-out.drv'... Running phase: unpackPhase Running phase: patchPhase Running phase: updateAutotoolsGnuConfigScriptsPhase Running phase: configurePhase no configure script, doing nothing Running phase: buildPhase error: linking with `cc` failed: exit status: 1 | = note: LC_ALL="C" PATH="/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/bin:/nix/store/a2pa8i6ns45r8mkan3nmj9j7k5v0smn4-rustc-wrapper-1.76.0/bin:/nix/store/z7abiwmqsg4m7d877sq1dinbxxsgg5jr-patchelf-0.15.0/bin:/nix/store/44di4gvw6lydf0hm9105palmlsdyvq5w-gcc-wrapper-13.2.0/bin:/nix/store/yw0pwdr8qm66l1b7s62zgczz37fg95ri-gcc-13.2.0/bin:/nix/store/a7zychff4w2w5ifjmli75h66adypvgh4-glibc-2.38-44-bin/bin:/nix/store/77fyfwmz29cz9j5x6yw2wrlm4rvasldv-coreutils-9.4/bin:/nix/store/bfvghacf729z1xnbkbznavv1312nshdz-binutils-wrapper-2.41/bin:/nix/store/f5xp690hwgqrvglkv61d4xcfqiijzmiq-binutils-2.41/bin:/nix/store/77fyfwmz29cz9j5x6yw2wrlm4rvasldv-coreutils-9.4/bin:/nix/store/ry6g1kym7g3i8813msq7b0gzqbdj1rfk-findutils-4.9.0/bin:/nix/store/5jpf44fy67dxhiwczcjc9w47hi96bm3q-diffutils-3.10/bin:/nix/store/jcpl9xd17v9c8aqkdwakhw3mymmagshp-gnused-4.9/bin:/nix/store/9fg6lk707pjy1k4hc3sx53lifhg1h09g-gnugrep-3.11/bin:/nix/store/w4b5hcaxh9jyr12bkms2i9kbksf7fax4-gawk-5.2.2/bin:/nix/store/f1w49nm88pnx65s4zw1azkxklndnw3sp-gnutar-1.35/bin:/nix/store/f79a7k1p7dc2cnkz0q5h6lqg48h1hbkc-gzip-1.13/bin:/nix/store/8h2cb2v7215i8awk2i1k702xcvr2d1zk-bzip2-1.0.8-bin/bin:/nix/store/qzi7n8bb148ccnkdkniq8m7y3jp37wvq-gnumake-4.4.1/bin:/nix/store/xz6h70zgmd6wf2931rdg5v4khnsxfg40-bash-5.2p26/bin:/nix/store/r7m8m864kbs5xsiw566fg2l64ya3v1zw-patch-2.7.6/bin:/nix/store/cfkpcy56gnr492mgnwrf3zpcqa9w1c1f-xz-5.6.0-bin/bin:/nix/store/1639yfqw64vivi4163ljawq8w8raypvj-file-5.45/bin" VSLANG="1033" "cc" "/build/rustcPgHRub/symbols.o" "main.main.506eb1eb2a190bce-cgu.0.rcgu.o" "main.3jowjigll6mjp1ta.rcgu.o" "-Wl,--as-needed" "-L" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd-68bd66fef6200e23.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libpanic_unwind-3c0633234349f35e.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libobject-ab93b99bdf00897c.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libmemchr-bd665da824976390.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libaddr2line-c36e128c53bf9cf5.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libgimli-b48ba914e2acbe38.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_demangle-43d08aea8e0de63a.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd_detect-378c0978269e65f0.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libhashbrown-2d5984e95ee70fe2.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-250ce9078ed6dd1d.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libminiz_oxide-f4fe08e31742e4fb.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libadler-2990c766846b3bd0.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libunwind-6ac3e54e8224c232.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcfg_if-bb3eb3ac424e49f6.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-bb6e2c2ca063bec3.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/liballoc-e0cf20cf65baa6b9.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_core-cf8a742adda6ebfa.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcore-d20d8104c5a385aa.rlib" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcompiler_builtins-e1803514747703e1.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/nix/store/1v1i7qfvq383bn390m39hynmba9bidsg-rustc-1.76.0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-o" "main" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs" = note: /build/rustcPgHRub/symbols.o: file not recognized: file format not recognized collect2: error: ld returned 1 exit status ```

With this change, I think it's clear this is unrelated to LDFLAGS overrides by qemu itself, but something different on how Nix changes environment variables when running on Ubuntu vs using Nix on NixOS.

I'm out of ideas after so many levels of ~yak shaving~ abstractions.

@bltavares Seems to be a regression introduced by QEMU 8.2. Most distros (and the setup-qemu-action for Github Actions) are on 8.1, while nixos-unstable updated to the 8.2 branch around 4 months ago (commit fc38d28b26904c294b33b64cdce0866e778b6bc8). If you pin nixpkgs to this branch of my fork (which partially reverts QEMU to 8.1.3) the build should succeed. I based that branch on your perl and qemu-user-static branches when I was trying to troubleshoot, but in theory this should work for dynamically linked QEMU as well. You can test this by pinning mainline nixpkgs to commit a45f095fced4dbafce7cc65c800b7c70302025ab (which should be the commit immediately preceding the update to 8.2), although I haven't tried that yet. Still unsure of the root cause, but based on the changelogs it appears to be either a /proc/cpuinfo issue or a vDSO issue.

Update: bisection indicates that the first bad commit is e8967b61, problem remains as of nixpkgs commit 82cbb284d39164690797fdf721070c54b6719a28 (QEMU 9.0.2).

NixOS / nixpkgs