rizinorg / cutter

Free and Open Source Reverse Engineering Platform powered by rizin
https://cutter.re
GNU General Public License v3.0
15.89k stars 1.15k forks source link

AppImage fails at dynamic linking stage on Ubuntu 16.04 (due to GLIBC 2.25 dependency) #1769

Closed assarbad closed 4 years ago

assarbad commented 5 years ago

Environment information

Describe the bug

Dynamic linker fails to link some symbol from the AppImage, as it expects GLIBC 2.25:

$ ./Cutter-v1.9.0-x64.Linux.appimage
./Cutter-v1.9.0-x64.Linux.appimage: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by ./Cutter-v1.9.0-x64.Linux.appimage)
./Cutter-v1.9.0-x64.Linux.appimage: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /tmp/.mount_CutterbjnZhc/usr/bin/../lib/libr_util.so)

To Reproduce

Steps to reproduce the behavior:

  1. Run on a Ubuntu 16.04
  2. Download the AppImage, e.g.: wget https://github.com/radareorg/cutter/releases/download/v1.9.0/Cutter-v1.9.0-x64.Linux.appimage
  3. Make AppImage executable, e.g. chmod +x ./Cutter-v1.9.0-x64.Linux.appimage
  4. Execute it and observe the issue described above, e.g.: ./Cutter-v1.9.0-x64.Linux.appimage

Expected behavior

Given this is an AppImage, I would expect it to run, especially since the kernel version it expects is rather ancient (2.6.18), but the GLIBC version is rather recent (February 2017).

$ file Cutter-v1.9.0-x64.Linux.appimage
Cutter-v1.9.0-x64.Linux.appimage: ELF 64-bit LSB executable, x86-64, version 1, dynamically linked, interpreter /lib64/l, for GNU/Linux 2.6.18, stripped

Of course certain optimizations for newer CPUs could still cause an issue, but the symbols should not.

Possible mitigation

As I wrote already on Telegram, the issue can be typically solved (i.e. keep building on a new system, but have that run on older GLIBC versions etc, provided the emitted opcodes are compatible with that older system). No need to revert back to a Xenial-based build, as that will keep the problem and simply shift it down a number of GLIBC versions.

One way is to define a macro in your C/C++ headers (e.g. in one included everywhere) like so:

#define GLIBC_COMPAT_SYMBOL(FFF) asm(".symver " #FFF "," #FFF "@GLIBC_2.2.5")

I think the idea may have originated here, but I am not sure. I've been using it for years and it only ever affected a handful of symbols in the software I was maintaining.

This will cause the symbol passed as FFF (a name, not a string!) to be aliased to a particular version of that symbol.

If you have ever built software for a number of platforms and GLIBC versions you may have run into memcpy@GLIBC_2.14 being required. Taking the above define you can state:

GLIBC_COMPAT_SYMBOL(memcpy);

... and it will be aliased to the "older" (but more sane in this case) version of memcpy.

The downside of this approach is that this has to be done in every single source file you have. By way of -include (when using gcc) this can be done.

That define provided above is specific to x86-64 ... because different architectures have been supported starting with different library versions, for example, you would have a different "common denominator" version. So for PowerPC 64-bit the above should use "@GLIBC_2.3" ... for x86-32 I have fared well using "@GLIBC_2.0". Other supported architectures may require other "lowest" version numbers.

Note however, that not all symbols are as trivial as memcpy (where 2.14 established the undefined behavior in case of overlapping buffers being passed).

Additionally you can - and that's generally the favorable method - tell the assembler what mapping to use (in case of the gcc driver frontend you'd pass):

gcc ... -Wa,--defsym,memcpy=memcpy@GLIBC_2.2.5 ...

... but this is still only viable if you don't statically link some pre-built component which itself has (newer) symbol requirements "imbued".

In that last case the only option (that I am aware of) left would be a linker script (also here) containing something using PROVIDE to alias one symbol name to another. This can be passed in such a way as to amend the existing implicit linker script or - using -T - to replace it. A wrapper like musl-gcc together with a customized specs file (see gcc -dumpspecs) could be used to automate this as much as possible without having to rely on Magic Sauce™ on the build system itself.

Anyway, the takeaway should be that the problem can be solved without reverting back to an older build system/compiler toolchain/...

assarbad commented 5 years ago

Alright, by way of --appimage-extract I extracted the squashfs contents.

Then I made myself a list of all the ELF files:

find -type f -exec file {} +|grep -P 'ELF \d{2}-bit'

I also cut myself a list of just the ELF file paths/names:

find -type f -exec file {} +|grep -P 'ELF \d{2}-bit'|cut -d : -f 1 > ../elf-names.txt

... and then ran all ELF files through readelf --dyn-syms to find any GLIBC requirements for GLIBC 2.10 and newer. For reference, GLIBC 2.10 was released in May 2009, ten years ago.

The outcome was a list of files I named recent-syms.txt (also attached):

cat ../elf-names.txt |while read fname; do echo "$fname"; readelf --dyn-syms "$fname"|grep -P 'GLIBC_2\.[12]\d'; done > ../recent-syms.txt

The summary is this

$ cat ../elf-names.txt |while read fname; do readelf --dyn-syms "$fname"|grep -P 'GLIBC_2\.[12]\d'; done|awk '$7 ~ /^UND/ {print $8}'|sort -u
accept4@GLIBC_2.10
clock_getres@GLIBC_2.17
clock_gettime@GLIBC_2.17
clock_nanosleep@GLIBC_2.17
clock_settime@GLIBC_2.17
__explicit_bzero_chk@GLIBC_2.25
explicit_bzero@GLIBC_2.25
__fdelt_chk@GLIBC_2.15
getauxval@GLIBC_2.16
getrandom@GLIBC_2.25
__longjmp_chk@GLIBC_2.11
memcpy@GLIBC_2.14
memfd_create@GLIBC_2.27
mkstemps@GLIBC_2.11
__ppoll_chk@GLIBC_2.16
prlimit64@GLIBC_2.13
pthread_setname_np@GLIBC_2.12
secure_getenv@GLIBC_2.17
setns@GLIBC_2.14

I didn't look any further as of yet. However, __fdelt_chk can be disposed of by way of undefining _FORTIFY_SOURCE and then defining it as zero:

-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0

... provided you build whatever component requires it (it's likely that the other *_chk functions will be gone as well).

clock_* functions could be linked from librt instead.

memcpy@GLIBC_2.14 can be easily diverted to point to the older (saner) version of memcpy (see this ticket for an in-depth discussion, in particular comment 38).

Anyway, that's quite a few functions to take care of. But it only has to be done once per update to a new build system.

Attached files

assarbad commented 5 years ago

I'll see if I can reproduce the AppImage build locally and thus send a PR or so.

scorpius commented 5 years ago

Have you made any progress?

assarbad commented 5 years ago

Have you made any progress?

@scorpius Nope, didn't look into this any further. The easiest way, moving forward, would likely be to include the more recent GLIBC in the AppImage. I can't see how that would be more problematic (license-wise) than the aggregate of what's already in there.

The only reason why that could fail on older systems would be system calls being used which are not available on older systems. But that's highly unlikely to happen as most of those have very very special use cases and wouldn't apply to something like r2 and Cutter with Python and what not ...

I am actually looking at techniques like the ones described right now to find if there's a way to recommend a least-intrusive way. So perhaps as a byproduct I may come up with something for here. But I wouldn't hold my breath. It's not one of my top priorities at the moment.

coolkingcole commented 4 years ago

I'm also currently having this problem on (4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64 GNU/Linux). Does anyone know a work around to get this working, I tried to ld preload the libraries, but that only solves one of the errors. I can preload radare2/libr/util/libr_util.so and it seems to link fine.

karliss commented 4 years ago

@coolkingcole I advice you to compile Cutter from source. For Appimage to work on older systems like Ubuntu 16.04 or Debian 9 it needs to be built against older glibc version. We would like to do that but our resources are limited. Hopefully someone will manage time to do that before those versions before end of maintenance for those versions.

assarbad commented 4 years ago

I think, btw, the stub is what causes the problem, not so much the contents. I still know too little about AppImage to help out, but if the stub could be replaced with one that statically links musl-libc, for example, this would definitely resolve the issue.