Getting segfault when trying to use libapprun_hooks

[probonopd]

In an effort to solve https://github.com/probonopd/go-appimage/issues/49, I was trying to use libapprun_hooks.so with an existing AppImage.

To reproduce:

1. Download https://timepirate.org/downloads/Commandoo.tar.gz and get the AppImage contained therein
2. Run the AppImage (it works)
3. Unpack the AppImage
4. Replace AppRun with the script below, make it executable
5. Download libapprun_hooks.so and put it in usr/lib64
6. Try to run

#!/bin/sh

HERE="$(dirname "$(readlink -f "${0}")")"
export APPDIR="$HERE"

MAIN=$(grep -r "^Exec=.*" "$HERE"/*.desktop | head -n 1 | cut -d "=" -f 2 | cut -d " " -f 1)
MAIN_BIN=$(find "$HERE/usr/bin" -name "$MAIN" | head -n 1)
LD_LINUX=$(find "$HERE" -name 'ld-*.so.*' | head -n 1)

echo "Run $MAIN_BIN from experimental self-contained bundle"

export XDG_DATA_DIRS="${HERE}"/usr/share/:"${XDG_DATA_DIRS}"
export QT_PLUGIN_PATH="${HERE}"/usr/lib/qt4/plugins/:"${HERE}"/usr/lib/i386-linux-gnu/qt4/plugins/:"${HERE}"/usr/lib/x86_64-linux-gnu/qt4/plugins/:"${HERE}"/usr/lib32/qt4/plugins/:"${HERE}"/usr/lib64/qt4/plugins/:"${HERE}"/usr/lib/qt5/plugins/:"${HERE}"/usr/lib/i386-linux-gnu/qt5/plugins/:"${HERE}"/usr/lib/x86_64-linux-gnu/qt5/plugins/:"${HERE}"/usr/lib32/qt5/plugins/:"${HERE}"/usr/lib64/qt5/plugins/:"${QT_PLUGIN_PATH}"
export INTERPRETER="${LD_LINUX}"
export LD_PRELOAD=libapprun_hooks.so

exec "${MAIN_BIN}" "$@"

Result:

me@host:~/squashfs-root$ ./AppRun 
Run /home/me/squashfs-root/usr/bin/commandoo from experimental self-contained bundle
Segmentation fault

What am I doing wrong?

[azubieta]

I got a 404 when trying to download the file you reference above.

Regarding your code, libapprun_hooks.so function is to remove AppImage specific variables when the process is forked by using of execv or some derivative.

You still need AppRun to resolve the right ld-linux, notice that you must specify the path to the system ld-linux, and the path to the bundle ld-linux. You can inspect an AppImage packaged with appimage-builder for details.

https://github.com/AppImageCrafters/appimage-manager/releases/download/v0.1.2/app-0.1.0-x86_64.AppImage

[probonopd]

You still need AppRun to resolve the right ld-linux,

How is it doing that? I was suspecting that it was done by LD_PRELOAD libapprun_hooks.so but apparently there is more to it?

[probonopd]

You can inspect an AppImage packaged with appimage-builder for details.

Tried doing so but noticed variables that I don't quite understand the meaning of yet, hence #6... ;-)

[azubieta]

I left some answers on #6, I'll also improve the README

[azubieta]

Closing it, please reopen if the problem persist.

[probonopd]

Here is a zipped AppDir that also segfaults for me. Possibly I am doing something wrong? Cannot figure it out by looking at strace.

https://transfersh.com/13S9o9/test.appdir.zip (available for 14 days)

It was made with -s -l deploy using https://github.com/probonopd/go-appimage/blob/master/src/appimagetool/releases/continuous plus adding AppRun and libapprun_hooks.so from this repository, plus an .env file manually.

By the way, I get the same(?) segfault if I replace the binary AppRun from this repository by the following script (is that even possible?):

HERE="$(dirname "$(readlink -f "${0}")")"
export APPIMAGE_UUID=1234567890
export SYSTEM_INTERP=/lib64/ld-linux-x86-64.so.2
export APPDIR_LIBC_VERSION=2.30
export APPDIR_INTERP=libc/lib64/ld-linux-x86-64.so.2
export RUNTIME_INTERP=/tmp/ld-1234567890.so
export EXEC_PATH=usr/bin/gedit
export EXEC_ARGS=
export APPDIR_LIBRARY_PATH=usr/lib64/:usr/lib64/gtk-3.0/3.0.0/immodules/:usr/lib64/gtk-3.0/3.0.0/printbackends/:usr/lib64/gtk-3.0/modules/
export LD_PRELOAD=$HERE/libapprun_hooks.so
exec "$EXEC_PATH" "$EXEC_ARGS" "$@"

[probonopd]

Can' make much sense of gdb either.

me@host:~$ unzip Downloads/test.appdir.zip 

me@host:~$ cd test.appdir/

me@host:~/test.appdir$ gdb AppRun 

(gdb) run

Starting program: /home/me/test.appdir/AppRun 
process 8529 is executing new program: /home/me/test.appdir/usr/bin/gedit
Program received signal SIGSEGV, Segmentation fault.

(gdb) bt full

#0  0x0000000000000000 in ?? ()
No symbol table info available.
#1  0x00007ffff7b35b0c in ?? ()
No symbol table info available.
#2  0x0000000000000000 in ?? ()
No symbol table info available.

(gdb) thread apply all bt full

Thread 1 (process 8529):
#0  0x0000000000000000 in ?? ()
No symbol table info available.
#1  0x00007ffff7b35b0c in ?? ()
No symbol table info available.
#2  0x0000000000000000 in ?? ()
No symbol table info available.

[probonopd]

Hold on, I just noticed that not all of the libc-related files have been put into the correct subdirectory. I am on it in go-appimage...

(Also, when I am trying to bundle mousepad rather than gedit, I do not get this segfault.)

Another thing I noticed: it makes a difference which one you do:

me@host:~$ ./test.appdir/AppRun 
me@host:~$ ( cd test.appdir ; ./AppRun )

(With a mouspad AppDir I am experimenting with this even makes the difference between running the app and doing nothing at all)

Apparently one needs to use $APPDIR in the .env file like this:

# Are comments with # and // allowed in this file?

# Where is this supposed to be coming from? Is it just a random value?
# Can't we default this to the md5hash of the path of the AppImage
# in the filesystem or something along those lines,
# hence removing the need for this line?
APPIMAGE_UUID=1234567890

SYSTEM_INTERP=/lib64/ld-linux-x86-64.so.2

# How to determine this programmatically from Go?
# Can't the lib figure this out itself, how much would the performance
# overhead be?
APPDIR_LIBC_VERSION=2.30

APPDIR_INTERP=$APPDIR/libc/lib64/ld-linux-x86-64.so.2

# Can't we default this to, e.g., /var/run/user/$UID/?
RUNTIME_INTERP=/tmp/ld-$APPIMAGE_UUID.so

# Can't we default this to Exec= from the toplevel desktop file?
EXEC_PATH=$APPDIR/usr/bin/gedit
# EXEC_ARGS

# Documentation unlear; should this contain all directories
# in which .so files reside in the AppImage? Including the special
# directories that we created for the libc familiy of libraries?
APPDIR_LIBRARY_PATH=$APPDIR/usr/lib64/:$APPDIR/usr/lib64/gtk-3.0/3.0.0/immodules/:$APPDIR/usr/lib64/gtk-3.0/3.0.0/printbackends/:$APPDIR/usr/lib64/gtk-3.0/modules/

# Can't we default this to libapprun_hooks.so?
LD_PRELOAD=libapprun_hooks.so

Attached is a new test AppDir in which I am using a bash AppRun to export the required variables:

https://transfersh.com/kiFQu/test-2.appdir.zip

I made this using go-appimage appimagetool -s -r deploy ... on Xubuntu 18.04.

When I run this on an older system, e.g., on Ubuntu 16.04, I am getting:

me@host:~$ '/home/me/Downloads/test.appdir/AppRun' 
/home/me/Downloads/test.appdir/usr/bin/gedit: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/../../../lib/x86_64-linux-gnu/libmount.so.1)
/home/me/Downloads/test.appdir/usr/bin/gedit: /lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.9' not found (required by /home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/gedit/.././libpng16.so.16)
/home/me/Downloads/test.appdir/usr/bin/gedit: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/../../../lib/x86_64-linux-gnu/./libblkid.so.1)
/home/me/Downloads/test.appdir/usr/bin/gedit: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/gedit/.././../../../lib/x86_64-linux-gnu/libexpat.so.1)
/home/me/Downloads/test.appdir/usr/bin/gedit: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/../../../lib/x86_64-linux-gnu/././libuuid.so.1)
/home/me/Downloads/test.appdir/usr/bin/gedit: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/gedit/.././../../../lib/x86_64-linux-gnu/./libsystemd.so.0)
/home/me/Downloads/test.appdir/usr/bin/gedit: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.27' not found (required by /home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/gedit/.././../../../lib/x86_64-linux-gnu/./libsystemd.so.0)
/home/me/Downloads/test.appdir/usr/bin/gedit: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/gedit/../././../../../lib/x86_64-linux-gnu/libbsd.so.0)
me@host:~$ ls  /home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/gedit/.././../../../lib/x86_64-linux-gnu/libexpat.so.1
/home/me/Downloads/test.appdir/usr/bin/../lib/x86_64-linux-gnu/gedit/.././../../../lib/x86_64-linux-gnu/libexpat.so.1

So it seems like the bundled libc is not used even though it is newer than the one on the target system. I must be missing something?

Ah, right, I need to add

APPDIR_LIBRARY_PATH=$APPDIR/usr/lib/x86_64-linux-gnu/:$APDIR/./lib/x86_64-linux-gnu/:$APPDIR/usr/lib64/

LIBC_LIBRARY_PATH=$APPDIR/libc/lib/x86_64-linux-gnu/:$APPDIRlibc/lib/x86_64-linux-gnu/:$APPDIR/libc/lib64/

(or so I'd assume). But as soon as I do that (in the .env file or in the bash AppRun version), I get:

Segmentation fault

strace then shows:

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe52566f000
writev(2, [{"/home/me/Downloads/test.appdir/u"..., 44}, {": ", 2}, {"/./lib/x86_64-linux-gnu/libz.so."..., 33}, {": ", 2}, {"version `ZLIB_1.2.9' not found ("..., 115}, {"\n", 1}], 6/home/me/Downloads/test.appdir/usr/bin/gedit: /./lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.9' not found (required by /home/me/Downloads/test.appdir/usr/lib/x86_64-linux-gnu/libpng16.so.16)
) = 197
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe52566e000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe52566d000
exit_group(1)                           = ?
+++ exited with 1 +++

Why is it not loading zlib from

me@host:~/Downloads/test.appdir$ find . | grep /libz.so.1
./libc/lib/x86_64-linux-gnu/libz.so.1

? Actually, it seems like quite many files are tried to be loaded from the system rather than from the AppImage, which is clearly entirely wrong:

me@host:~/Downloads/test.appdir$ strace ./AppRun 2>&1 | grep '"/./'open("/./lib/x86_64-linux-gnu/tls/x86_64/libapprun_hooks.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/./lib/x86_64-linux-gnu/tls/x86_64", 0x7ffdcb04edb0) = -1 ENOENT (No such file or directory)
open("/./lib/x86_64-linux-gnu/tls/libapprun_hooks.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/./lib/x86_64-linux-gnu/tls", 0x7ffdcb04edb0) = -1 ENOENT (No such file or directory)
open("/./lib/x86_64-linux-gnu/x86_64/libapprun_hooks.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/./lib/x86_64-linux-gnu/x86_64", 0x7ffdcb04edb0) = -1 ENOENT (No such file or directory)
open("/./lib/x86_64-linux-gnu/libapprun_hooks.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/./lib/x86_64-linux-gnu", {st_mode=S_IFDIR|0755, st_size=5470, ...}) = 0
open("/./lib/x86_64-linux-gnu/libgedit.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/./lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libz.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libresolv.so.2", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libmount.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/liblzma.so.5", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/librt.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libblkid.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/./lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libdbus-1.so.3", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libexpat.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libuuid.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libsystemd.so.0", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libbsd.so.0", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libgcrypt.so.20", O_RDONLY|O_CLOEXEC) = 3
open("/./lib/x86_64-linux-gnu/libgpg-error.so.0", O_RDONLY|O_CLOEXEC) = 3
writev(2, [{"/home/me/Downloads/test.appdir/u"..., 44}, {": ", 2}, {"/./lib/x86_64-linux-gnu/libz.so."..., 33}, {": ", 2}, {"version `ZLIB_1.2.9' not found ("..., 115}, {"\n", 1}], 6/home/me/Downloads/test.appdir/usr/bin/gedit: /./lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.9' not found (required by /home/me/Downloads/test.appdir/usr/lib/x86_64-linux-gnu/libpng16.so.16)

[probonopd]

Once instance in which the AppRun segfault happens is when the .env file contains unreadable information, e.g,. a line of random text.

[probonopd]

As per https://github.com/AppImageCrafters/AppRun/issues/11#issuecomment-679281570, the bash AppRun that I have so far is not sufficient. So let's focus on the .env for the binary AppRun for now. Any ideas @azubieta?

[azubieta]

So it seems like the bundled libc is not used even

AppRun guess wich glib to use by reading the libc.so file.

strings /lib/x86_64-linux-gnu/libc-2.27.so | grep GLIBC_

If the system glib version is greater than the bundled one the you set LD_LIBRARY_PATH to $APPDIR_LIBRARY_PATH:$LD_LIBRARY_PATH.

if the bundled glib version is greater then set LD_LIBRARY_PATH to $LIBC_LIBRARY_PATH:$APPDIR_LIBRARY_PATH:$LD_LIBRARY_PATH.

I guess that I should add this to the documentation too.

Notice that you will have to bundle grep, cut and other command you use in your bash script. There will be race conditions with those commands and they dependencies. That's why I created a statically linked binary for the AppRun.

[probonopd]

I see. Do you understand why this AppDir fails with the statically linked binary for the AppRun?

[ghost]

Hi @azubieta and @probonopd , please see my solution to this problem at:

https://github.com/probonopd/go-appimage/issues/49

[probonopd]

Wdyt @azubieta? I still think we should find a solution that removes the need to copy ld-linux to /tmp... kinda like what https://github.com/probonopd/libhookexecv does for WINE.

[azubieta]

Prefix ld-linux on each execve call has a nasty side effect, it makes /proc/self/exec point to ld-linux instad of the executed binary. If we could make /proc/self/exec report the right path a lot of issues will be solved.