autoPatchelfHook fails with libmpi.dbg

[rodarima]

The debug symbols in libmpi.so for Intel MPI are distributed in a separate file libmpi.dbg with an (wrong?) ELF header. For what I can tell, the auto-patchelf hook finds any file and attempts to determine if it has a valid executable ELF header.

The .dbg file has an incorrect header for readelf:

$ readelf -h libmpi.dbg 
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x3ee40
  Start of program headers:          64 (bytes into file)
  Start of section headers:          252744 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         8
  Size of section headers:           64 (bytes)
  Number of section headers:         30
  Section header string table index: 29
readelf: Error: the dynamic segment offset + size exceeds the size of the file

Also the patchelf attempt fails:

...
searching for dependencies of /nix/store/br931h7y37rl4kshm5nhgqixk6303ii7-intel-mpi-2019.7.217/lib/libmpi.dbg
strange: no string table
builder for '/nix/store/ca1l0r1gzsj4jb84zc6dhgs4kqfgqxm4-intel-mpi-2019.7.217.drv' failed with exit code 1
error: build of '/nix/store/ca1l0r1gzsj4jb84zc6dhgs4kqfgqxm4-intel-mpi-2019.7.217.drv' failed

I was looking for a way to exclude this file from autoPatchelfHook. Can we just patch files with the executable bit set adding -executable to the find filters? So I could easily chmod -x libmpi.dbg to exclude it.

diff --git a/pkgs/build-support/setup-hooks/auto-patchelf.sh b/pkgs/build-support/setup-hooks/auto-patchelf.sh
index 72970623ed7..7a799d23fcd 100644
--- a/pkgs/build-support/setup-hooks/auto-patchelf.sh
+++ b/pkgs/build-support/setup-hooks/auto-patchelf.sh
@@ -216,7 +216,7 @@ autoPatchelf() {
           [ -n "$(echo "$segmentHeaders" | grep "^ *INTERP\\>")" ] || continue
       fi
       autoPatchelfFile "$file"
-    done < <(find "$@" ${norecurse:+-maxdepth 1} -type f -print0)
+    done < <(find "$@" ${norecurse:+-maxdepth 1} -type f -executable -print0)
 }

 # XXX: This should ultimately use fixupOutputHooks but we currently don't have

[stale[bot]]

I marked this as stale due to inactivity. → More info

[colinxs]

I ran into the same issue with debug files while attempting to package julia-bin. Rather than checking if the file is executable (in my case they are), which would require manually chmod'ing, could we do something along of the lines of: 1) Allow specifying a list of files for autoPatchElf to ignore. 2) And/or rather than exiting on error, print a warning and continue.

[colinxs]

For (2), this change was sufficient:

--- a/pkgs/build-support/setup-hooks/auto-patchelf.sh
+++ b/pkgs/build-support/setup-hooks/auto-patchelf.sh
@@ -189,6 +189,16 @@ addAutoPatchelfSearchPath() {
             \( -name '*.so' -o -name '*.so.*' \) -print0)
 }

+# TODO need to first check that the DYNAMIC section exists before
+# extracting the size.
+hasEmptyDynamic() {
+    file="$1"
+    hex='0[xX][0-9a-fA-F]\{16\}'
+    line1='DYNAMIC[ \t0-9A-z]*'
+    dynsize="$(readelf -l "$file" | grep "$line1" -A1 | grep -v "$line1" | grep -o "$hex" | head -n 1)"
+    [ "$dynsize" = "0x0000000000000000" ]
+}
+
 autoPatchelf() {
     local norecurse=

@@ -217,6 +227,7 @@ autoPatchelf() {

     while IFS= read -r -d $'\0' file; do
       isELF "$file" || continue
+      hasEmptyDynamic "$file" && continue
       segmentHeaders="$(LANG=C $READELF -l "$file")"
       # Skip if the ELF file doesn't have segment headers (eg. object files).
       # not using grep -q, because it can cause Broken pipe

Which just parses the output of readelf to get the size of the DYNAMIC section. If it's 0/empty, then skip, a) because there's nothing to do, but b) because patchelf can't handle empty DYNAMIC sections.

I think you could similarly get rid of the "strange: no string table" messages from auto-patchelf.sh and elsewhere from calling patchelf on ELF's with empty DYNAMIC sections.

Maybe a cleaner option is for patchelf to exit silently without error if the dynamic section is empty (not sure if that's a feature or bug).

Tagging @DavHau since you seem to familiar with this code :).

[stale[bot]]

I marked this as stale due to inactivity. → More info

[cf4f67]

I also have problem with autopatchelf choking on debug files

https://github.com/NixOS/patchelf/issues/373

https://github.com/yvt/nixpkgs/commit/49be2c828875522289e2250307e85c48625bccc7#commitcomment-70690616