Linux AppImage build fails if run as root

[freakboy3742]

Reported by @axel-kah

If you run a briefcase create linux build while logged in as root, it attemps to build the Docker container with HOST_UID and HOST_GID set to 0. This causes the useradd step to fail:

Step 12/16 : ARG HOST_UID
 ---> Running in 063a424d6a86
Removing intermediate container 063a424d6a86
 ---> 4657e96c180d
Step 13/16 : ARG HOST_GID
 ---> Running in a59336d3cdb7
Removing intermediate container a59336d3cdb7
 ---> 4fede95af7ee
Step 14/16 : RUN groupadd --gid $HOST_GID briefcase || true
 ---> Running in bb37131e7a73
groupadd: GID '0' already exists
Removing intermediate container bb37131e7a73
 ---> a94d6f409c76
Step 15/16 : RUN useradd --uid $HOST_UID --gid $HOST_GID brutus --home /home/brutus
 ---> Running in 55dc8b2957d8
useradd: UID 0 is not unique
The command '/bin/sh -c useradd --uid $HOST_UID --gid $HOST_GID brutus --home /home/brutus' returned a non-zero code: 4

Error building Docker container for helloworld.

This is because the default user in Docker is root, with UID 0.

We need to either: a. Modify the docker container to be resilient to UIDs that already exist, or b. Add error handling to prevent briefcase from being invoked with users that already exist.

[axel-kah]

Exploring option a)

Seems like useraddand usermod both feature the --non-unique option to allow duplicate UIDs. When adding this option in the modified template, referenced in my pyproject.toml, then I could get away with this and briefcase create executes without errors. A subsequent briefcase build will bail out with the familiar problem that linuxdeploy is known to not being executable in containers (conditions apply?) linuxdeploy/linuxdeploy/issues/86. Applying the suggested fix (replacing the magic bytes) and chowning the appimage with my local useraccount will produce a working tutorial app. If I leave the appimage with its root ownership, then the appimage does not work when run as root with "Run Once" showing an error message:

Failed to register AppImage in AppImageLauncherFS: could not open map file

Not sure if that is a problem with the created app image or with the AppImageLauncher run as root.

[freakboy3742]

Are you sure the "not executable" problem is actually the same bug as the one you're reporting? We're already using a different workaround that doesn't require modifying the magic bytes - see AppImage/AppImageKit#828. I suspect the problem might have more to do with the chowning process - we've got potentially 4 users involved here (user ID and root, both inside and outside the Docker container), so it's easy to imagine there will be a conflict there.

[axel-kah]

Spent some more time on option a).

It seems like there are two separate docker vs AppImage issues at play here which was not entirely obvious when first reading all the docker related AppImage / linuxdeploy issues. They also are more fundamental in nature and have nothing to do with briefcase or its Dockerfile per se:

1. Failing to execute any AppImage inside docker because of missing fuse. Error message will look like this:

   
   root@05f88012bf95:/apps# ./python3.8.3-cp38-cp38-manylinux1_x86_64.AppImage --version
   dlopen(): error loading libfuse.so.2

AppImages require FUSE to run. You might still be able to extract the contents of this AppImage if you run it with the --appimage-extract option. See https://github.com/AppImage/AppImageKit/wiki/FUSE for more information

2. Failing to execute any AppImage inside docker because of magic bytes. Error message will look like this:

root@4326b1d4c5c5:/.briefcase/tools# ./python3.8.3-cp38-cp38-manylinux1_x86_64.AppImage
bash: ./python3.8.3-cp38-cp38-manylinux1_x86_64.AppImage: No such file or directory

The first one can be worked around with by using the `--appimage-extract-and-run` switch or setting the `APPIMAGE_EXTRACT_AND_RUN` env variable.

The second one doesn't seem to even get to parsing cli flags or env variables because the issue seems rooted in the magic bytes which are evaluated before any flags by the applications logic. This issue goes away when modifying the magic bytes as suggested in https://github.com/AppImage/AppImageKit/issues/828#issue-344623665.

I can reproduce issue#1 with Ubuntu 20.04 and 18.04 running inside a VM, installing latest docker-ce and running a bare Ubuntu 16.04 container as root like so:
`docker run -it -v /home/osboxes/apps:/apps ubuntu:16.04 /bin/bash`

I can reproduce issue#2 running Linux Mint 19.3 (based on Ubuntu 18.04) natively on my Laptop with the same bare Ubuntu 16.04 container. Another user describes the same problem here https://github.com/AppImage/AppImageKit/issues/1027

__EDIT:__ When running LM19.3 inside a VM (same kernel version, same docker version) I get issue#1 instead of issue#2 :confused: .

[axel-kah]

I've asked the maintainers of AppImage in https://github.com/AppImage/AppImageKit/issues/1027 if they see any downside in tweaking the magic bytes. Depending on their response we could also apply that workaround when the build is run in a docker context.

Then the Dockerfiles useradd could be extended with the aforementioned --non-unique flag to have a linux build flow that is robust to existing UIDs.

Now this userstory is missing two more chapters. Invoking briefcase as sudo/root and what to do with the build result.

If the user want to build as root (as in sudo su), then briefcase will behave as expected. If the user wants to build as non-root (but is worried about the security implications of adding the user to the docker group) then he will run briefcase create/briefcase build and it will fail because docker requires root permissions. Most people will try sudo briefcase create but that will fail because the venv is lost when using sudo. UX bummer I'd say. What does work is sudo bash -c "source beeware-venv/bin/activate && briefcase create" but that is not intuitive for most users. briefcase could catch dockers "permission denied" error and give an example of how to call it via sudo correctly.

The rest is not strictly part of the primary build issue anymore but kinda belongs to the briefcase userstory none the less. Once the build is finshed (either by root or sudo) the resulting AppImages, which are owned by root:root can be executed by non-root users without problems. Executing AppImages with elevated permissions obviously works inside the headless docker container, but does not work on linux desktops and is being discussed here: https://github.com/TheAssassin/AppImageLauncher/issues/266 I see the ball in the AppImage* corner here. briefcase documentation could only add a note to invoke briefcase with sudo and not as root, which implicitly (and hopefully) avoids the situation that the AppImage is unintentionally executed with elevated privileges.

@freakboy3742 Oppinions?

[freakboy3742]

I'm a little confused about your diagnosis regarding Issue #1 - because Briefcase is already using --appimage-extract-and-run.

As for the use of sudo: frankly, "didn't work; run with sudo" is a behavioral antipattern, and I'm not inclined to do anything to accommodate or encourage that behavior. Briefcase needs Docker to operate; if a sysadmin is willing to give someone access to sudo as root, but not access to the Docker group... well... I don't know what to say - but I'm definitely not inclined to encourage the behavior. If we're able to identify that we're running as sudo and raise an error, that would be good.

It sounds like error handling around the edge case where the user isn't in the Docker group is definitely called for, though. The situation where a user hasn't been added to the Docker group is the sort of thing that could easily occur; identifying that edge case and suggesting a course of action (i.e., contact your sysadmin and get added to the Docker group) is the least we can do.

The one thing I'm unclear on at this point is whether genuine "run as root" works (i.e., logged in as root, or sudo su). Does that still require the --non-unique flag (or any other modifications) in the Dockerfile?

[axel-kah]

I'm a little confused about your diagnosis regarding Issue #1 - because Briefcase is already using --appimage-extract-and-run.

Sorry about the confusion. It is true that briefcase is already mitigating issue#1 by using --appimage-extract-and-run. I just wanted to make clear that there is another issue that can't be mitigated by the same workaround which is not obvious from reading AppImage/AppImageKit#828 which should also be addressed by briefcase IMHO.

The one thing I'm unclear on at this point is whether genuine "run as root" works (i.e., logged in as root, or sudo su). Does that still require the --non-unique flag (or any other modifications) in the Dockerfile?

I just double checked on that fresh Ubuntu 18.04 VM that it does require the --non-unique flag. It's then possible to briefcase {new,create,build} logged in as root without problem.

As for the use of sudo: frankly, "didn't work; run with sudo" is a behavioral antipattern, and I'm not inclined to do anything to accommodate or encourage that behavior. Briefcase needs Docker to operate; if a sysadmin is willing to give someone access to sudo as root, but not access to the Docker group... well... I don't know what to say - but I'm definitely not inclined to encourage the behavior. If we're able to identify that we're running as sudo and raise an error, that would be good.

Understood, but I don't think we can detect sudo. If the user calls sudo briefcase we get a sudo: briefcase: command not found which suggests that the sudo process does not inherit the shell environment and hence would have to activate the venv before calling briefcase but how should sudo know that?

It sounds like error handling around the edge case where the user isn't in the Docker group is definitely called for, though. The situation where a user hasn't been added to the Docker group is the sort of thing that could easily occur; identifying that edge case and suggesting a course of action (i.e., contact your sysadmin and get added to the Docker group) is the least we can do.

What do you think is the most robust approach to identifiying that edge case? Evaluating docker returncodes, matching parts of docker error messages or going straight to calling groups in a subprocess and checking if docker is mentioned?

[freakboy3742]

If there's a unique return code, that's probably the best option; if the return code isn't unique, supplementing that with looking for an error message in output is probably advisable. Digging into groups is probably overkill. As long as we can catch the fact that the Docker step failed, and if possible, show an error message that hints at the likely cause and solution, that should be sufficient.