Fails for applications that use user namespaces

[BentonEdmondson]

My native unshare works:

$ unshare -U echo hi
hi
$ echo $?
0

But the AppImage doesn't:

$ ./bundle util-linux /bin/unshare
$ ./unshare-x86_64.AppImage -U echo hi
unshare-x86_64.AppImage: unshare failed: Operation not permitted
$ echo $?
1

I believe this is the reason Steam, Chromium, and Electron apps don't work:

$ ./steam-x86_64.AppImage
bwrap: No permissions to creating new namespace, likely because the kernel does not allow non-privileged user namespaces. On e.g. debian this can be enabled with 'sysctl kernel.unprivileged_userns_clone=1'.

$ ./ungoogled-chromium-122.0.6261.69-x86_64.AppImage
[52161:52161:0226/172919.336302:FATAL:setuid_sandbox_host.cc(158)] The SUID sandbox helper binary was found, but is not configured correctly. Rather than run without sandboxing I'm aborting now. You need to make sure that /nix/store/zamhhinmqjvvxj1ipfijvrpqs85q3rc0-ungoogled-chromium-122.0.6261.69-sandbox/bin/__chromium-suid-sandbox is owned by root and has mode 4755.
Trace/breakpoint trap (core dumped)

$ ./bitwarden-2024.2.0-x86_64.AppImage
[60941:0226/173049.236563:FATAL:setuid_sandbox_host.cc(158)] The SUID sandbox helper binary was found, but is not configured correctly. Rather than run without sandboxing I'm aborting now. You need to make sure that /nix/store/9zw7gk36f07lnn6y60ksd2s063q46ri1-electron-28.2.2/libexec/electron/chrome-sandbox is owned by root and has mode 4755.
Trace/breakpoint trap (core dumped)

This article suggests that this is not a fundamental limitation of the AppImage format; assuming unprivileged user namespaces are enabled in the kernel, Electron AppImages can be run without disabling their sandbox.

[ralismark]

nix-appimage uses unprivileged user namespaces itself -- it works by copying all required nix store files into the AppImage, then "mounting" them at /nix/store. I don't see any other way of doing that without user namespaces.

I'm guessing that you're getting these errors because Linux doesn't allow nested user namespaces? If that's the case I'm leaning towards marking this as WONTFIX, sorry :(

[BentonEdmondson]

No, nesting user namespaces is possible:

$ unshare -c unshare -U echo hi
hi

I looked at it more closely and I found the problem. The man page says you can't create a user namespace within a chroot:

  EPERM (since Linux 3.9)
         CLONE_NEWUSER was specified in flags and the caller is in
         a chroot environment (i.e., the caller's root directory
         does not match the root directory of the mount namespace
         in which it resides).

You chroot right here: https://github.com/ralismark/nix-appimage/blob/17dd6001ec228ea0b8505d6904fc5796d3de5012/apprun.c#L188-L198

When I remove those lines the problem seems to be completely fixed. What is the reason for the chroot?

[ralismark]

i need chroot in order to mount the bundled /nix/store. on systems without nix, I'm pretty sure i need there to already be a /nix directory in order to mount, and since i can't make that directory, i need to make a copy of / with the extra directory and chroot

[caspervk]

Is it possible to use the mountroot as a lowerdir in an overlayfs instead of chrooting to it? This would allow "creating" (overlaying) a /nix directory even though the mountpoint doesn't exist.

On nix systems the existing nix store will be merged and take precedence (as the upperdir). I don't know if this could be a problem. Maybe the AppImage's nix store needs to be made content-addressed so potential collisions are irrelevant?

[ralismark]

@caspervk I had a look at replacing the current "recreate /" method with using overlayfs, which would likely be much nicer. I'm pretty sure overlayfs only ever modifies upperdir, so you'd be looking to overlay / on top of the bundled nix/store. I had a play around with trying to get this to work but there are some major problems:

• The biggest one is that overlayfs requires the upperdir and the workdir to be on the same filesystem. There isn't a portable way to create a suitable workdir.
• Mounts are also not propagated through to the overlayfs, which breaks a lot of systems.

I've started looking at mergerfs/unionfs-fuse/etc. I think ideally the solution would involve using a FUSE merging filesystem, then chrooting into it, which would avoid needing user namespaces entirely. Unfortunately, I don't really have the time to fully investigate this, so if anyone wants to look into it, here's my progress so far:

• As related work, NixOS has to do something similar for FHSEnv. They have a chroot method (which is the same as our "recreate /" method) and a bubblewrap method. bubblewrap and proot might work for what we want but neither directly support overlaying our bundled nix/store without simply listing everything. Which might be okay, I haven't looked into it.
• mergerfs seems to do what we want with mergerfs -o cache.files=partial,dropcacheonclose=true,category.create=mfs underlay=RO:/ mountpoint (where underlay is a directory with our bundled nix/store). However, "mergerfs is designed and intended to be run as root and may exibit incorrect behavior if run otherwise", which basically means this won't work.
• However, that means it's likely possible to make a FUSE filesystem that does do what we want (it'd probably also be useful for NixOS), it'd basically forward all file operations to the real /, and when the target file doesn't exist, check our bundled files. I have zero experience with actually implementing FUSE things though, so no clue how difficult this would be.