Read-only ELF .llvm_stackmaps is incompatible with PIE

[vext01]

Take the following input programs:

$ cat no-stackmaps.ll 
define void @main() noinline {
        %1 = alloca i32
        ret void
}

$ cat stackmaps.ll
declare void @llvm.experimental.stackmap(i64, i32, ...)

define void @main() noinline {
        %1 = alloca i32
    call void (i64, i32, ...) @llvm.experimental.stackmap(i64 0, i32 0, ptr %1)
        ret void
}

Using LLVM as of the time of writing:

$ clang --version
clang version 18.0.0git (https://github.com/llvm/llvm-project 607f19cb947a25fe7ed131d983a90961f3c2541a)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/vext01/research/llvm-project/build/bin

If you use lld to link and emit a stackmap section, the main symbol becomes undefined.

gnu ld, no stackmaps

$ clang -Wno-override-module no-stackmaps.ll && nm a.out | grep main | grep -v start
0000000000001130 T main

gnu ld, with stackmaps

$ clang -Wno-override-module stackmaps.ll && nm a.out | grep main | grep -v start
/usr/bin/ld: /tmp/stackmaps-7c8644.o: warning: relocation against `main' in read-only section `.llvm_stackmaps'
/usr/bin/ld: warning: creating DT_TEXTREL in a PIE
0000000000001130 T main

lld, no stackmaps

$ clang -Wno-override-module -fuse-ld=lld  no-stackmaps.ll && nm a.out | grep main | grep -v start
00000000000016b0 T main

lld, with stackmaps

$ clang -Wno-override-module -fuse-ld=lld  stackmaps.ll && nm a.out | grep main | grep -v start
                 U main

Why is this? Is this bug?

This breaks something I'm working on that uses dladdr(3) at runtime.

CC @lhames

[vext01]

If you turn off PIE, then the symbol comes back:

$ clang -fno-PIE -no-pie -Wno-override-module  -fuse-ld=lld  stackmaps.ll && nm a.out | grep main | grep -v start
0000000000201640 T main

[MaskRay]

ELF .llvm_stackmaps is readonly. The format seems not designed for static links using -pie/-shared because dynamic relocations are needed. It can switch to PC-relative relocations to avoid this problem.

% readelf -Wr stackmaps.o

Relocation section '.rela.llvm_stackmaps' at offset 0x150 contains 1 entry:
    Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
0000000000000010  0000000400000001 R_X86_64_64            0000000000000000 main + 0

Relocation section '.rela.eh_frame' at offset 0x168 contains 1 entry:
    Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
0000000000000020  0000000200000002 R_X86_64_PC32          0000000000000000 .text + 0

There will be a linker error if you specify -z text or use binutils configured with --enable-textrel-check=yes.

lld creates a canonical PLT entry, so we see an undefined symbol with a non-zero st_value.

% readelf -Ws a.out | grep 'Num:\| main'
   Num:    Value          Size Type    Bind   Vis      Ndx Name
   Num:    Value          Size Type    Bind   Vis      Ndx Name
    22: 0000000000001720     0 FUNC    GLOBAL DEFAULT  UND main

However, the symbol is not exported and we will get a R_X86_64_JUMP_SLOT referencing the null symbol. I have created https://github.com/llvm/llvm-project/pull/75095 to add the error to lld, but .llvm_stackmaps also needs a change to work with -pie.

[vext01]

Thanks for the info.

By the way, I notice some other "possibly useful at runtime" sections, aren't marked SHT_ALLOC, which means that the loader won't load them, and all addresses inside talk about file offsets instead of virtual addresses. Look for example at the basic block labels section (.llvm_bb_addr_map, created with -lto-basic-block-sections=labels).

One possible fix (maybe not the best) is to do the same with the stackmap section (i.e. revoke SHT_ALLOC) at the cost of:

• the consumer having to read the section from the ELF file on-disk (because it isn't loaded).
• the consumer having to query the loader (to find the base load address) if they want to get virtual addresses out of the stackmaps.

I must admit, I don't know much about relocations, but I will give the link you posted a read.