containers / bubblewrap

Low-level unprivileged sandboxing tool used by Flatpak and similar projects
Other
3.97k stars 237 forks source link

Bwrap seems to fail in any ARM64 installation #651

Open Whistlerone opened 3 months ago

Whistlerone commented 3 months ago

I'm trying to use bwrap as part of a build process, and it works fine right now on ubuntu 20.04 and 22.04. However all versions of bwrap are failing on the ARM64 versions of 20.04 and 22.04. failure no.1 is: running bwrap without root permissions results in bwrap: capset failed: Operation not permitted even for bwrap --version this is not the case on x86.

failure no.2 is: running any command with sudo results in: bwrap: Creating new namespace failed: Invalid argument even a basic command like bwrap --bind ./root/ / --new-session bash

this applies to both the package repo versions(0.4.0 and 0.6.1 respectively) as well as the latest build from github(0.10.0)

none of these issues appear on x86 machines.

rusty-snake commented 3 months ago

It does work for me with bwrap 0.8.0 and Linux 6.10.4-200.fc40.aarch64. To me this sound like ubuntus arm64 kernels miss some features.

smcv commented 3 months ago

"Fails in my ARM64 installation" is not the same thing as "fails in any ARM64 installation".

running bwrap without root permissions results in bwrap: capset failed: Operation not permitted

This sounds like you have installed bwrap setuid root, but you are running in some sort of confined environment (container? AppArmor? ...) that doesn't allow a setuid-root bwrap to work as intended.

In particular, if you're running on an an Ubuntu 24.04 kernel, there are AppArmor restrictions that prevent bwrap from working as intended unless it is given a special AppArmor profile. bwrap cannot solve this, this is something that you have to solve via AppArmor configuration. You should see AppArmor denials logged in the audit log or the system log (systemd Journal) if it's this.

even for bwrap --version

That shouldn't happen for bwrap ≥ 0.1.8, which checks whether its arguments are exactly bwrap --version before it calls into any code paths that can call capset. Are you sure you're running the bwrap executable that you think you're running?

running any command with sudo results in: bwrap: Creating new namespace failed: Invalid argument

This usually means the kernel isn't allowing bwrap to do what it needs to do, for one of several poorly-documented reasons (see user_namespaces(7) for some possibilities). bwrap cannot solve this, it needs to be run in an environment where creating new user namespaces is possible.

Are you trying to run bwrap inside a chroot environment? The kernel doesn't allow this (because if it did, you'd be able to use it to access files outside the chroot) and I think the error code in this case is EINVAL.

Or, are you trying to run bwrap inside a container or similar "not the real machine" environment?

smcv commented 3 months ago

none of these issues appear on x86 machines

Are you running the x86 and ARM versions identically, or are they differently-configured? (Possible differences include: kernel version; features enabled in the kernel; in a container or not; in a chroot or not; different container technologies; being emulated or not; AppArmor, SELinux or no LSM; LSM in enforcing mode or not; ...)

Whistlerone commented 3 months ago

I know that 24.04 has issues, that's why I am testing with 20.04 and 22.04. I am using Docker containers with apparmor=unconfined. But I want to reiterate, I am running the containers with the same dockerfile and the same run flags. but I'm only having these issues with containers that are running with ARM64 images

smcv commented 2 months ago

I am using Docker containers with apparmor=unconfined. But I want to reiterate, I am running the containers with the same dockerfile and the same run flags. but I'm only having these issues with containers that are running with ARM64 images

On what kernel, host OS and machine (presumably x86?) are you running the x86 images?

On what kernel, host OS and machine (x86 with emulation? ARM64?) are you running the ARM64 images?