[lld] add warning for unfindable .so library needed by current .so

[CoTinker]

demo

cat > def.s <<e
e

cat > ref.s <<e
.globl fun
fun:
  bl foo@PLT
e

cat > main.s <<e
.globl _start
_start:
  bl fun@PLT
e

llvm-mc -filetype=obj -triple=aarch64 main.s -o main.o
llvm-mc -filetype=obj -triple=aarch64 def.s -o def.o && ld.lld -shared def.o -o def.so
llvm-mc -filetype=obj -triple=aarch64 ref.s -o ref.o && ld.lld -shared ref.o -o ref.so def.so

ld

ld will report warning:

% ld main.o ref.so

ld: warning: def.so, needed by ref.so, not found (try using -rpath or -rpath-link)
ld: ref.so: undefined reference to `foo'

lld

No alarm is generated when lld is linking.

% lld main.o ref.so

I think the behavior of ld is more reasonable. Can we add alarms for lld?

[CoTinker]

@MaskRay

[llvmbot]

@llvm/issue-subscribers-lld-elf

Author: Longsheng Mou (CoTinker)

demo ``` #--- def.s #--- ref.s .globl fun fun: bl foo@PLT #--- main.s .globl _start _start: bl fun@PLT % llvm-mc -filetype=obj -triple=aarch64 main.s -o main.o % llvm-mc -filetype=obj -triple=aarch64 def.s -o def.o && ld.lld -shared def.o -o def.so % llvm-mc -filetype=obj -triple=aarch64 ref.s -o ref.o && ld.lld -shared ref.o -o ref.so def.so ``` # ld ld will report warning: ``` % ld main.o ref.so ld: warning: def.so, needed by ref.so, not found (try using -rpath or -rpath-link) ld: ref.so: undefined reference to `foo' ``` # lld No alarm is generated when lld is linking. ``` % lld main.o ref.so ``` I think the behavior of ld is more reasonable. Can we add alarms for lld?

[MaskRay]

I think this is a wont-implement: the value of this diagnostic is low and its behavior is dubious. This is related to ld(2) -rpath-link. It has some interop with --no-allow-shlib-undefined and --as-needed.

"The linker uses the following search paths to locate required shared libraries:" with 12 bullet points (complex rules). I think it is dubious to have so many rules for the linker to emulate a loader, which can never be perfect. gold made the decision to ignore -rpath-link and lld follows its steps. Linkers for other object file formats do not implement the search rules, either.

https://maskray.me/blog/2021-06-13-dependency-related-linker-options

First, complexity. In the GNU ld manpage, -rpath-link=dir lists 12 rules, which are just overwhelming. Several points don't play well with cross linking. Eliminating difference between native link and a cross link is an important design choice of ld.lld. In addition, I know --no-copy-dt-needed-entries can affect the behavior subtly, which isn't even documented.

Second, performance. Recursive loading needs to parse additional shared objects which may slow down the link a bit.

Third, practicality. --allow-shlib-undefined is an unfortunate default for -shared. Changing it may be disruptive today. Mach-O and PE/COFF have many problems but this may be a place where they got right. I have several paragraphs describing it in https://maskray.me/blog/2021-06-13-dependency-related-linker-options Recursive loading of --allow-shlib-undefined gives me a feeling of working around the wrong default.

[hstk30-hw]

About performance, in the cross compiltion situation, it error/warning untill dyn load step （on board） if not reported in compile step. It wastes more time actually.

https://github.com/llvm/llvm-project/blob/f779ec7c13bdfccd29655d13a325f34c60797a76/lld/ELF/Writer.cpp#L1798C1-L1801C16

Besides, there are some problems when do the --no-allow-shlib-undefined check only after loaded all the needed share objests.

1. Some share object added by compiler default, the deps of those share object should guaranteed by compiler.
2. Some share object deps soName have version, once the share object upgrade, we must change -lxx (may be soft link can solve it), but still not so smart.

Anyway, maybe Community not planed to do it, I will. And can you give some suggestion about it. @MaskRay

"The linker uses the following search paths to locate required shared libraries:" with 12 bullet points (complex rules). I think it is dubious to have so many rules for the linker to emulate a loader, which can never be perfect. gold made the decision to ignore -rpath-link and lld follows its steps. Linkers for other object file formats do not implement the search rules, either.

We don't necessary to implement like ld about the complex rules rpath-link.