drrun glibc 2.34 __libc_start_main dependence doesn't run on older distros

[Desperado17]

Trying to cross-compile dynamorio from source for linux arm using the arm32 toolchain file.

cmake -DCMAKE_TOOLCHAIN_FILE=../dynamorio/make/toolchain-arm32.cmake ../dynamorio

However, the result always links against glibc 2.34 and it drrun won't start without it. The target has only glibc 2.31 installed and I cannot change that.

nm lists this as problematic reference:

U __libc_start_main@GLIBC_2.34

Same problem, different symbol as here: https://github.com/DynamoRIO/dynamorio/issues/5474

Target was built on an X64 Ubuntu 22.04 with arm cross compilers installed. Should be gcc 11.

[derekbruening]

Unfortunately, this is a general problem caused by glibc versioned symbols. Since we're not likely to roll our own startup code I think there are only two options:

1. Try asm(.symver...) -- but if that has be in front of every use it's not clear how to apply it to __libc_start_main?
2. Require that a pre-glibc-2.34 toolchain (or the cross-toolchain here) be used to build binaries meant to be run on pre-glibc-2.34 target systems. If such a toolchain can't be easily installed from repositories the user (or our GA CI or Jenkins builders in the future if we upgrade them to 2.34) would have to build from sources or something.

You could try to work around it by not using drrun: make a config file manually or on another machine and directly exec libdynamorio.so. But that's not very user-friendly.

Not sure there are any other solutions -- unless someone knows of one?

[derekbruening]

Note that this is not specific to DynamoRIO or drrun in any way: anyone building any software using glibc will have this problem and for __libc_start_main the symver route is not clear: so I would imagine there are other projects complaining about this new 2.34 symbol too.

[Desperado17]

@derekbruening See last comment here https://groups.google.com/g/dynamorio-users/c/kqh5Jl_H4LA

I don't think this is a test, is it?

[xdje42]

Data point: I found the patch that introduced the problem: https://sourceware.org/bugzilla/show_bug.cgi?id=23323 It's apparently part of security hardening.

The comment "A new symbol version __libc_start_main@@GLIBC_2.34 is introduced because new binaries running on an old libc would not run their ELF constructors, leading to difficult-to-debug issues." suggests this would be nasty to work around. IIUC, there is a fundamental change in the API b/w the crt0.o linked into the binary and libc.

[derekbruening]

It is looking like the best solution may be to switch to a different libc such as musl. We may want to consider doing the same for client tools to solve the #5437 private loader problems with new glibc/pthread/ld.so developments as well: glibc is really making it hard for us to use their libraries.

[derekbruening]

Xref https://groups.google.com/g/dynamorio-users/c/qwzjnfXwcPA/m/ikYgE3SEAgAJ?utm_medium=email&utm_source=footer where this is printed as:

CMake Error at /home/jenkins/DR/dynamorio/core/CMake_readelf.cmake:165 (message):
  *** Error:
  /home/jenkins/DR/release/build_debug-64/bin64/drconfig
  has too-recent import(s):

     GLOBAL DEFAULT  UND _[...]@GLIBC_2.34

And -wide is needed on the readelf command to print the full symbol name -- someone should add that permanently.