Make AppImages run on Alpine

[probonopd]

Alpine has libc6-compat which provides ld-linux, so in theory it should run there, but in practice we are getting:

/patchelf-0.10 # apk add libc6-compat
(1/1) Installing libc6-compat (1.1.22-r3)
OK
/patchelf-0.10 # ./appimagetool-390-x86_64.AppImage 
Error relocating ./appimagetool-390-x86_64.AppImage: gnu_dev_makedev: symbol not found

[probonopd]

Documentation says:

Conforming to The makedev(), major(), and minor() functions are not specified in POSIX.1, but are present on many other systems. Notes These interfaces are defined as macros. Since glibc 2.3.3, they have been aliases for three GNU-specific functions: gnu_dev_makedev(), gnu_dev_major(), and gnu_dev_minor(). The latter names are exported, but the traditional names are more portable.

[TheAssassin]

Would a statically linked binary (e.g., using musl internally) work on every system? Can we safely assume that? I mean, we have a dependency on FUSE, but that's more API than ABI, and any FUSE client should be able to talk to any distro, no matter what libc they use.

[TheAssassin]

CC #877

[CosmicToast]

There is a library named libgcompat that provides most of the additional (non-standard) features in glibc for musl-based systems. When I was trying to do some fairly specific stuff it got me a bit further in the process (ncurses would act weird), but it should be most of the solution here - https://code.foxkit.us/adelie/gcompat (There's also a GitHub mirror)

I think the optimal solution will eventually be to build things against musl (ideally statically) and package all the deps in the appimage. That way it'll even run on "older" distributions, and has less of an associated cost (due to the size and scope of musl and co.). Appimages will always take up more space than "native" counterparts, but it makes up for it immensely using everything it brings in - leaning into that aspect seems natural, especially given how cheap storage is, relative to everything else (within reasonable bounds, but I do encourage commenters to-be to look at their filesystem usage patterns).

[CosmicToast]

Would a statically linked binary (e.g., using musl internally) work on every system? Can we safely assume that?

That's a fairly safe assumption. ABI incompatibilities are a thing, as well as kernel headers, but I've done some fairly extreme things in the past (such as statically linked shells and utilities for use in a system that hasn't been updated in about a decade). Musl allowing true static linking is very important relative to glibc in that use-cases.

[TheAssassin]

The problem with glibc is that we need the right ld-linux as linker, too, right? I don't know exactly how musl works in detail.

I think we need to finally extract and separate the runtime from the rest of the code, then we can create the regular builds with their semi-static linking and also experiment with truly static runtime builds.

[probonopd]

Some instinct tells me that sooner or later we'll end up making the runtime a version of ld-linux, so that it doesn't have to rely on one from the system... (musl's is even MIT licensed)

[CosmicToast]

The problem with glibc is that we need the right ld-linux as linker, too, right? I don't know exactly how musl works in detail.

That gcompat project I mentioned provides a linker (compatibility stub*) too, but yes, that's part of it.

I think we need to finally extract and separate the runtime from the rest of the code, then we can create the regular builds with their semi-static linking and also experiment with truly static runtime builds. Some instinct tells me that sooner or later we'll end up making the runtime a version of ld-linux, so that it doesn't have to rely on one from the system... (musl's is even MIT licensed)

Both of these sound interesting. Also, consider me available for general help (pings and such) - I believe the general concept of AppImages to be the best way for general linux distribution, for a variety of reasons. I'm also a maintainer of a few packages on Alpine Linux and the co-founder of Abyss OS - in which we plan to use the general AppImage idea (not necessarily the main implementation, it's a bit early to decide on that) to distribute most user-facing applications (by way of packaging them ourselves), so I'm somewhat invested as it is :slightly_smiling_face:.

I'm fairly busy, but knowing where and what to poke at to understand the codebase would be useful as well, if possible (as much as I abhor C).

[TheAssassin]

@5paceToast thanks for your offer. We've already found enough reasons to make a AppImage type 3, in which we intend to solve a lot of problems. Your input will be welcome.

[TheAssassin]

I've started a draft, comments welcome: https://github.com/TheAssassin/type3-runtime

(Please continue the discussion there, here it's off topic.)

[probonopd]

I don't think we need a new type in order to address the points mentioned in this thread; this imho all can be achieved by a new runtime for type 2 images (i.e., a squashfs or other filesystem appended to an ELF binary).

[wenerme]

About one year now, any progress here to support musl ?

[probonopd]

Looks like no one is working on it.

[vinniec]

Has anyone tried with this? https://github.com/sgerrand/alpine-pkg-glibc I tried without a positive result but I actually don't know if it should have worked or not.

[anki-code]

Hi @vinniec! Thank you for the link. Yes! It works!

I've made a test with the xonsh shell and report the successfull result - https://github.com/sgerrand/alpine-pkg-glibc/issues/153#issuecomment-795334536

[mensinda]

There is also https://github.com/mensinda/libRuntime, which will give you a statically linked runtime that can be passed to appimagetool with --runtime-file.

However, this won't really make a difference if the actual packaged program also depends on glibc, since the host version of the c library is used in general. The way I work around this is by also packaging musl or glibc (including ld-linux.so) and then launching LD_LIBRARY_PATH=.../AppDir/usr/lib /path/to/ld-linux.so .../AppDir/usr/bin/myprog.

[probonopd]

@mensinda do you have such an AppImage for download somewhere? I'd like how it fares on helloSystem (FreeBSD).

[mensinda]

@probonopd Yes, I have some "AppImages" here. For reference, they are as a demonstration for https://github.com/mesonbuild/meson/pull/7518.

I am not sure if they really are AppImages according to the spec, since I am packaging python3, ninja, pkgconf, and Meson. Also, the "AppImage" self extracts when an actual Meson project is configured.

These binaries are built in the latest Alpine docker container and the complete c library is packaged (including the musl *ld-linux.so). However, instead of relying on LD_LIBRARY_PATH (which messes stuff up for Meson, since we call other binaries outside the AppDir) I modified the rpath instead.

I have successfully tested the 64bit binary on Ubuntu 10.04 (not that you can actually do anything useful there).

[toniz4]

Any reason why this doesn't work? I'm missing a package or something like that?

/tmp λ curl -LO 'https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   632  100   632    0     0   1468      0 --:--:-- --:--:-- --:--:--  1466
100 2121k  100 2121k    0     0  1677k      0  0:00:01  0:00:01 --:--:-- 7954k
/tmp λ chmod +x appimagetool-x86_64.AppImage
/tmp λ ./appimagetool-x86_64.AppImage
This doesn't look like a squashfs image.

Cannot mount AppImage, please check your FUSE setup.
You might still be able to extract the contents of this AppImage
if you run it with the --appimage-extract option.
See https://github.com/AppImage/AppImageKit/wiki/FUSE
for more information
open dir error: No such file or directory
/tmp λ ldd appimagetool-x86_64.AppImage
        /lib64/ld-linux-x86-64.so.2 (0x7fc91d075000)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7fc91d075000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7fc91d075000)
        libz.so.1 => /lib/libz.so.1 (0x7fc91d05b000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fc91d075000)
Error relocating appimagetool-x86_64.AppImage: gnu_dev_makedev: symbol not found
/tmp λ patchelf --add-needed libgcompat.so.0 appimagetool-x86_64.AppImage
/tmp λ ldd appimagetool-x86_64.AppImage
        /lib64/ld-linux-x86-64.so.2 (0x7f829bb73000)
        libgcompat.so.0 => /lib/libgcompat.so.0 (0x7f829bb61000)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7f829bb73000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f829bb73000)
        libz.so.1 => /lib/libz.so.1 (0x7f829bb47000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f829bb73000)
        libucontext.so.1 => /lib/libucontext.so.1 (0x7f829bb42000)
        libobstack.so.1 => /usr/lib/libobstack.so.1 (0x7f829bb3d000)
/tmp λ ./appimagetool-x86_64.AppImage
This doesn't look like a squashfs image.

Cannot mount AppImage, please check your FUSE setup.
You might still be able to extract the contents of this AppImage
if you run it with the --appimage-extract option.
See https://github.com/AppImage/AppImageKit/wiki/FUSE
for more information
open dir error: No such file or directory
/tmp λ

Unlike libc6-compat, gcompat defines gnu_dev_makedev, so why it doesn't work?

[probonopd]

Great question. I don't know the answer, sorry.

[xproot]

xproot@cedric:~/AppImages$ ./browservice-v0.9.2.2-aarch64.AppImage 
bash: ./browservice-v0.9.2.2-aarch64.AppImage: No such file or directory
xproot@cedric:~/AppImages$ ls
browservice-v0.9.2.2-aarch64.AppImage

Is this like a well known alpine issue?

[CosmicToast]

xproot@cedric:~/AppImages$ ./browservice-v0.9.2.2-aarch64.AppImage 
bash: ./browservice-v0.9.2.2-aarch64.AppImage: No such file or directory
xproot@cedric:~/AppImages$ ls
browservice-v0.9.2.2-aarch64.AppImage

Is this like a well known alpine issue?

This is a well known linux issue. Exec failures are reported this way. Usually, it's a consequence of wrong architecture or missing libraries/symbols - normally you'd try running it through file(1) and ldd(1).

However, in this case, it's likely using the original stub, which dynamically links to glibc (which is obviously not present), which is indeed what this (unresolved, unlikely to be resolved) bug is about. Similarly, you would find running musl-dynlinked binaries a challenge on a glibc distribution without installing musl.

[toniz4]

xproot@cedric:~/AppImages$ ./browservice-v0.9.2.2-aarch64.AppImage 
bash: ./browservice-v0.9.2.2-aarch64.AppImage: No such file or directory
xproot@cedric:~/AppImages$ ls
browservice-v0.9.2.2-aarch64.AppImage

Is this like a well known alpine issue?

I get this when not using gcompat. Using gcompat i get the error shown above.

[s-zeid]

I've also opened https://git.adelielinux.org/adelie/gcompat/-/issues/349 for this.

[probonopd]

If I understand it right, it Looks like https://github.com/eth-cscs/spack-batteries-included is providing a solution for this. Should we backport these changes into the AppImage runtime?

Differences and improvements over AppImage runtime spack.x uses zstd for faster decompression; spack.x itself is an entirely static binary; spack.x does not need to dlopen libfuse.so

Reference: https://github.com/AppImage/AppImageKit/issues/1120#issuecomment-1060331710 cc @haampie

[probonopd]

I believe the AppImages from https://github.com/probonopd/go-appimage/releases/tag/continuous which are using the experimental static runtime might work on Alpine (if fusermount is installed and the fuse kernel module is loaded) but I fail to get DHCP and internet access running when booting into

https://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/alpine-standard-3.12.0-x86_64.iso

Does anyone know how to test this?

[s-zeid]

On v3.12, setup-interfaces, press enter a few times for defaults, and service networking restart. On ~newer Alpine versions~ v3.15 or later, you can just do setup-interfaces -r.

Also, you may want to setup repositories: setup-apkrepos -1, and (if desired) uncomment community in /etc/apk/repositories then apk update.

[s-zeid]

And appimagetool from your first link does run on my Alpine edge system.

[probonopd]

Discussion continues in

• https://github.com/AppImage/AppImageKit/issues/877

[xplshn]

If I understand it right, it Looks like https://github.com/eth-cscs/spack-batteries-included is providing a solution for this. Should we backport these changes into the AppImage runtime?

Differences and improvements over AppImage runtime spack.x uses zstd for faster decompression; spack.x itself is an entirely static binary; spack.x does not need to dlopen libfuse.so

Reference: #1120 (comment) cc @haampie

It uses the same method as flatpaks, runtimes contain a small Linux system... In this case, it contains gtar, python3, binutils and other misc programs and libraries that were compiled against glibc but then modified to link against a local glibc (a glibc installed inside of the runtime)

[xplshn]

If only the guides for packaging AppImages told developers to ALWAYS TRY to statically link using musl-gcc or an Alpine container or install... AppImages would be smaller and would be actually portable.

[probonopd]

Everything has upsides and downsides. But yes, I like the approach you suggest.