llvmbot
commented
1 month ago @llvm/issue-subscribers-lld-elf
Author: None (chitao1234)
# Brief
When cross compiling, ld.lld `-m` option overwritten by host machine linker script, binutils ld handle this situation correctly.
# Detailed problem explanation
In a Linux system using glibc, `/lib/libc.so` is a linker script that sets `OUTPUT_FORMAT` to the host machine.
Example libc.so:
```
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /usr/lib/libc.so.6 /usr/lib/libc_nonshared.a AS_NEEDED ( /usr/lib/ld-linux-x86-64.so.2 ) )
```
When cross compiling using clang and lld, if set `-L` option incorrectly, lld would search host library first and load the linker script, such that lld would fail with a weird error message.
```
% ld.lld --sysroot=/usr/aarch64-linux-gnu -EL --hash-style=gnu --build-id --eh-frame-hdr -m aarch64linux -pie -dynamic-linker /lib/ld-linux-aarch64.so.1 -o a.out /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/Scrt1.o /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/crti.o /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/crtbeginS.o -L/lib -L/usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0 -L/usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/../lib64 -L/usr/aarch64-linux-gnu/lib/../lib64 -L/usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib -L/usr/aarch64-linux-gnu/lib --verbose test.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/crtendS.o /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/crtn.o
ld.lld: set maximum concurrency to 16, specify --threads= to change
ld.lld: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/Scrt1.o
ld.lld: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/crti.o
ld.lld: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/crtbeginS.o
ld.lld: test.o
ld.lld: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a
ld.lld: /lib/libgcc_s.so
ld.lld: /lib/libgcc_s.so.1
ld.lld: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a
ld.lld: /lib/libc.so
ld.lld: /usr/lib/libc.so.6
ld.lld: /usr/lib/libc_nonshared.a
ld.lld: /usr/lib/ld-linux-x86-64.so.2
ld.lld: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a
ld.lld: /lib/libgcc_s.so
ld.lld: /lib/libgcc_s.so.1
ld.lld: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a
ld.lld: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/crtendS.o
ld.lld: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/crtn.o
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/Scrt1.o is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/crti.o is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/crtbeginS.o is incompatible with elf64-x86-64
ld.lld: error: test.o is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_1_1.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_2_1.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_4_1.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_8_1.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_16_1.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_1_2.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_2_2.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_4_2.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_8_2.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_16_2.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_1_3.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_2_3.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_4_3.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_8_3.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_16_3.o) is incompatible with elf64-x86-64
ld.lld: error: /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/libgcc.a(cas_1_4.o) is incompatible with elf64-x86-64
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
```
It's worth noting that binutils ld handle this case correctly, due to it being single targeted.
binutils ld behavior:
```
% aarch64-linux-gnu-ld --sysroot=/usr/aarch64-linux-gnu -EL --hash-style=gnu --build-id --eh-frame-hdr -m aarch64linux -pie -dynamic-linker /lib/ld-linux-aarch64.so.1 -o a.out /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/Scrt1.o /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/crti.o /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/crtbeginS.o -L/lib -L/usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0 -L/usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/../lib64 -L/usr/aarch64-linux-gnu/lib/../lib64 -L/usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib -L/usr/aarch64-linux-gnu/lib test.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/crtendS.o /usr/bin/../lib64/gcc/aarch64-linux-gnu/14.1.0/../../../../aarch64-linux-gnu/lib/crtn.o
aarch64-linux-gnu-ld: skipping incompatible /lib/libgcc_s.so.1 when searching for libgcc_s.so.1
aarch64-linux-gnu-ld: skipping incompatible /lib/libgcc_s.so.1 when searching for libgcc_s.so.1
aarch64-linux-gnu-ld: skipping incompatible /lib/libc.so when searching for -lc
aarch64-linux-gnu-ld: skipping incompatible /lib/libc.a when searching for -lc
aarch64-linux-gnu-ld: skipping incompatible /lib/libgcc_s.so.1 when searching for libgcc_s.so.1
aarch64-linux-gnu-ld: skipping incompatible /lib/libgcc_s.so.1 when searching for libgcc_s.so.1
```
Currently in lld, `-m` option is used by clang to set target, `-m` sets the `config->emulation`, but linker script sets `config->bfdname`, lld would always prefer using `bfdname` as the target, casuing the aforementioned cryptic error message.
This situation is quite common unfortuantely, due to some build tool not handling cross compilation well enough (like `libtool`). It would be nice for lld to behave the same as binutils ld in this situation, both for compatibility and robustness.
# Additional infomation:
## Versions:
```
LLD 18.1.8 (compatible with GNU linkers)
clang version 18.1.8
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
GNU ld (GNU Binutils) 2.42.0
Copyright (C) 2024 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
```
Linux distro: EndeavourOS
# Steps for reproducing:
```shell
# linker error
echo 'int main(){}' | clang -target aarch64-linux-gnu -xc --sysroot=/usr/aarch64-linux-gnu -L/lib -fuse-ld=lld -
# no error
echo 'int main(){}' | clang -target aarch64-linux-gnu -xc --sysroot=/usr/aarch64-linux-gnu -L/lib -
```
Brief
When cross compiling, ld.lld
-moption overwritten by host machine linker script, binutils ld handle this situation correctly.Detailed problem explanation
In a Linux system using glibc,
/lib/libc.sois a linker script that setsOUTPUT_FORMATto the host machine.Example libc.so:
When cross compiling using clang and lld, if set
-Loption incorrectly, lld would search host library first and load the linker script, such that lld would fail with a weird error message.It's worth noting that binutils ld handle this case correctly, due to it being single targeted. binutils ld behavior:
Currently in lld,
-moption is used by clang to set target,-msets theconfig->emulation, but linker script setsconfig->bfdname, lld would always prefer usingbfdnameas the target, casuing the aforementioned cryptic error message.This situation is quite common unfortuantely, due to some build tool not handling cross compilation well enough (like
libtool). It would be nice for lld to behave the same as binutils ld in this situation, both for compatibility and robustness.Additional infomation:
Versions:
Linux distro: EndeavourOS
Steps for reproducing: