systemd's udevadm fails in extra-utils

[RossComputerGuy]

Describe the bug

Inside of the extra-utils derivation, udevadm fails to execute. Outside of extra-utils, udevadm can execute. This issue occurs with and without any arguments. Strace reveals the exec syscall fails.

execve("/nix/store/l3g7si329816p93wwi8q05cihkryg35n-extra-utils/bin/udevadm", ["/nix/store/l3g7si329816p93wwi8q0"...], 0xffffedefdf30 /* 79 vars */) = -1 EINVAL (Invalid argument)
+++ killed by SIGSEGV +++
[1]    746261 segmentation fault (core dumped)  strace /nix/store/l3g7si329816p93wwi8q05cihkryg35n-extra-utils/bin/udevadm

Dumping the binary with llvm-readelf -d shows this:

Dynamic section at offset 0x89600 contains 32 entries:
  Tag                Type           Name/Value
  0x000000000000001d (RUNPATH)      Library runpath: [/nix/store/l3g7si329816p93wwi8q05cihkryg35n-extra-utils/lib]
  0x0000000000000001 (NEEDED)       Shared library: [libsystemd-shared-256.so]
  0x0000000000000001 (NEEDED)       Shared library: [libblkid.so.1]
  0x0000000000000001 (NEEDED)       Shared library: [libpthread.so.0]
  0x0000000000000001 (NEEDED)       Shared library: [libc.so.6]
  0x0000000000000001 (NEEDED)       Shared library: [ld-linux-aarch64.so.1]
  0x000000000000001e (FLAGS)        BIND_NOW
  0x000000006ffffffb (FLAGS_1)      NOW PIE
  0x0000000000000015 (DEBUG)        0x0
  0x0000000000000007 (RELA)         0x7b38
  0x0000000000000008 (RELASZ)       41784 (bytes)
  0x0000000000000009 (RELAENT)      24 (bytes)
  0x000000006ffffff9 (RELACOUNT)    1611
  0x0000000000000017 (JMPREL)       0x11e70
  0x0000000000000002 (PLTRELSZ)     14040 (bytes)
  0x0000000000000003 (PLTGOT)       0xa9958
  0x0000000000000014 (PLTREL)       RELA
  0x0000000000000006 (SYMTAB)       0xd0000
  0x000000000000000b (SYMENT)       24 (bytes)
  0x0000000000000005 (STRTAB)       0x46c4
  0x000000000000000a (STRSZ)        13423 (bytes)
  0x000000006ffffef5 (GNU_HASH)     0x46a8
  0x0000000000000019 (INIT_ARRAY)   0x8af10
  0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
  0x000000000000001a (FINI_ARRAY)   0x8af18
  0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
  0x000000000000000c (INIT)         0x78a34
  0x000000000000000d (FINI)         0x78a4c
  0x000000006ffffff0 (VERSYM)       0x4098
  0x000000006ffffffe (VERNEED)      0x45b4
  0x000000006fffffff (VERNEEDNUM)   4
  0x0000000000000000 (NULL)         0x0

Steps To Reproduce

Steps to reproduce the behavior:

1. Build NixOS with systemd.package = pkgs.pkgsLLVM.systemd; and boot.initrd.systemd.package = pkgs.pkgsLLVM.systemd;.

Expected behavior

NixOS init stage 1 should complete

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

Notify maintainers

@flokli @Kloenk

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

---

Add a :+1: reaction to issues you find important.

[flokli]

Can you provide some more context on what you're trying to do and what NixOS option you set?

I assume you're talking about the extraUtils in nixos/modules/system/boot/stage-1.nix, but that's a derivation providing some additional utilities to be present in the stage1. It's not executed in stage1 itself, but during the build, and invoking udevadm in there cannot work (as the build is sandboxed).

But maybe I'm mis-guessing things, so please provide some context on what you're trying accomplish (the bigger picture), and how (config).

[RossComputerGuy]

Config: https://github.com/RossComputerGuy/nixpkgs-llvm-ws/blob/master/nixos/default.nix

During stage 1 init, several segfaults occur. Adding the trace option, the udevadm command appears to segfault repeatedly. Running udevadm on the host from the extra init derivation also crashes but the udevadm command in systemd appears to work. I think this is happening because strip ran on the binary twice. This issue only occurs with an LLVM built NixOS system.

[RossComputerGuy]

Here's a disassembly of the program https://termbin.com/g0um

Tried using gdb.

Reading symbols from /nix/store/cb9140939brgj80d7dxgzyx7pi49my34-extra-utils/bin/udevadm...
(No debugging symbols found in /nix/store/cb9140939brgj80d7dxgzyx7pi49my34-extra-utils/bin/udevadm)
(gdb) run
Starting program: /nix/store/cb9140939brgj80d7dxgzyx7pi49my34-extra-utils/bin/udevadm
During startup program terminated with signal SIGSEGV, Segmentation fault.
(gdb) info registers
The program has no registers now.
(gdb) disassemble
No frame selected.
(gdb) disassemble main
No symbol table is loaded.  Use the "file" command.
(gdb) disassemble __libc_start_main
Dump of assembler code for function __libc_start_main@plt:
   0x0000000000078a90 <+0>: adrp    x16, 0xa9000
   0x0000000000078a94 <+4>: ldr x17, [x16, #2424]
   0x0000000000078a98 <+8>: add x16, x16, #0x978
   0x0000000000078a9c <+12>:    br  x17
End of assembler dump.
(gdb) disassemble 

Seems something did change which caused this segfualt: https://termbin.com/5612

[flokli]

I'm not sure this is something specific to the way we package systemd in nixpkgs, more like a LLVM-related issue… So might not be the right person to ask about this.

[RossComputerGuy]

Testing with this overlay if it fixes the issue:

systemd.override {
  stdenv = stdenv.override {
    cc = stdenv.cc.override {
      inherit (gnuStdenv.cc) bintools;
    };
  };
}