llvmbot
commented
10 months ago @llvm/issue-subscribers-lld-coff
Author: Martin Storsjö (mstorsjo)
LTO builds of FFmpeg on Windows on x86 fail, because LLD believes that the link has unresolved symbols, so it errors out before proceeding to the actual LTO compile (where those symbols would appear). This has been reported to FFmpeg at https://trac.ffmpeg.org/ticket/10548. The original, problematic piece of code can be viewed at https://github.com/FFmpeg/FFmpeg/blob/n6.1/libavcodec/x86/mlpdsp_init.c.
The issue can be reproduced with a very minimal reduced case that looks like this:
```c
extern char func_pos;
static const void *const jumptable[] = { &func_pos };
void func(void) {
__asm__ volatile(
"func_pos:\n"
:: "r"(jumptable)
);
}
```
Attempting to build and link it with LTO:
```
$ clang -target x86_64-windows-msvc -c lto-repro.c -flto
$ lld-link lto-repro.o -out:lto-repro.exe -entry:func -subsystem:console
lld-link: error: undefined symbol: func_pos
>>> referenced by lto-repro.c
>>> lto-repro.o
```
Without the option `-flto`, linking succeeds.
The same experiment also succeeds with Clang when targeting Linux:
```
$ clang -target x86_64-linux-gnu -c lto-repro.c -flto
$ ld.lld lto-repro.o -o lto-repro.exe -e func
```
The same experiment also succeeds with GCC, for both Linux and Windows targets.
The root issue is that the inline assembly block, within the function `func`, defines a symbol `func_pos`, which the C code outside of it refers to, but the LTO object file does not indicate that the symbol `func_pos` will be defined by the inline assembly:
```
$ clang -target x86_64-windows-msvc -c lto-repro.c -flto
$ llvm-nm lto-repro.o
---------------- T func
U func_pos
---------------- d jumptable
```
This looks exactly the same when Clang targets Linux as well.
The reason why this fails on Windows, is that before starting the actual LTO compilation, lld-link checks whether there are unresolved symbols, to be able to print errors to the developer immediately if it looks like the link is going to fail, instead of having to wait for the full LTO compilation to complete, just to print errors: https://github.com/llvm/llvm-project/blob/llvmorg-17.0.6/lld/COFF/Driver.cpp#L2374-L2379 In the ELF version of LLD, there's nothing corresponding (probably because most linked ELF objects are allowed to have undefined symbols?).
These links do succeed if just the pre-LTO symbol availability check is skipped.
In order to facilitate symbol discovery from LTO objects, the LLVM LTO object file implementation does actually peek into the inline assembly statements, but only for module level inline assembly, not for function internal inline assembly:
```c
__asm__(
"static_func:\n"
".globl extern_func\n"
"extern_func:\n"
" call undefined_sym\n"
);
void func(void) {
__asm__(
"inline_static_func:\n"
".globl inline_extern_func\n"
"inline_extern_func:\n"
" call inline_undefined_sym\n"
);
}
```
```
$ clang -target x86_64-linux-gnu -c lto-symbols.c -flto
$ llvm-nm lto-symbols.o
---------------- T extern_func
---------------- T func
---------------- t static_func
U undefined_sym
```
Here, both the defined and undefined symbols referenced in the module level inline assembly are picked up, but not the ones from the function level inline assembly.
CC @MaskRay
kasper93
rnk
mstorsjo
amyspark
lhmouse
LTO builds of FFmpeg on Windows on x86 fail, because LLD believes that the link has unresolved symbols, so it errors out before proceeding to the actual LTO compile (where those symbols would appear). This has been reported to FFmpeg at https://trac.ffmpeg.org/ticket/10548. The original, problematic piece of code can be viewed at https://github.com/FFmpeg/FFmpeg/blob/n6.1/libavcodec/x86/mlpdsp_init.c.
The issue can be reproduced with a very minimal reduced case that looks like this:
Attempting to build and link it with LTO:
Without the option
-flto, linking succeeds.The same experiment also succeeds with Clang when targeting Linux:
The same experiment also succeeds with GCC, for both Linux and Windows targets.
The root issue is that the inline assembly block, within the function
func, defines a symbolfunc_pos, which the C code outside of it refers to, but the LTO object file does not indicate that the symbolfunc_poswill be defined by the inline assembly:This looks exactly the same when Clang targets Linux as well.
The reason why this fails on Windows, is that before starting the actual LTO compilation, lld-link checks whether there are unresolved symbols, to be able to print errors to the developer immediately if it looks like the link is going to fail, instead of having to wait for the full LTO compilation to complete, just to print errors: https://github.com/llvm/llvm-project/blob/llvmorg-17.0.6/lld/COFF/Driver.cpp#L2374-L2379 In the ELF version of LLD, there's nothing corresponding (probably because most linked ELF objects are allowed to have undefined symbols?).
These links do succeed if just the pre-LTO symbol availability check is skipped.
In order to facilitate symbol discovery from LTO objects, the LLVM LTO object file implementation does actually peek into the inline assembly statements, but only for module level inline assembly, not for function internal inline assembly:
Here, both the defined and undefined symbols referenced in the module level inline assembly are picked up, but not the ones from the function level inline assembly.
CC @MaskRay