Host your own docker image

[Rojikku]

Greetings. I maintain https://aur.archlinux.org/packages/dangerzone so people in the Arch based distributions can potentially install dangerzone. It worked fine until podman. I thought it was because I setup podman wrong. I looked further into it upon user complaint, and the issue is actually because you want me to build the container image.

You provide a nice script. I get it. But it doesn't work in chroots. Most installers that use PKGBUILDs build the package in chroots, and I try to write things to work in chroots. I have spent two and a half hours attempting to get this to build in a chroot, and I cannot. The podman community does not have a solution thus far, nor does google.

Resolution: Please build and host your own dockerimage. Github and Dockerhub will both happily use your existing dockerfile to do this. It should cost you nothing. I do not believe it is possible to build a docker container in a chroot unless you configure it in some way. I don't think that's going to work very well at all on a variety of systems. It's just not good. Hosting your own image solves this problem.

Possible outcomes:

1. Above, and hopefully the build works and all is good. You may have to push an update getting rid of the image-id.txt requirement.
2. I can build and host it myself, but I have absolutely no interest in maintaining docker images, so I drop this package
3. Provide me a method with which to build via a PKGBUILD, and I will try it. If it works, sure, but having each user build their own image still hurts me.

Alternatively, if you ask, I will be happy to disown this package- enabling anyone who wishes to become the maintainer (I.E. you, since you probably know what you're doing).

Also this will solve #152

[legoktm]

We are currently discussing basically the same issue in preparation for packaging Dangerzone for Debian (and hopefully that'll make it available for downstreams like Tails too): https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=986856#29

Building and shipping such a large container is impractical and also has technical blockers, whether it's the chroot issue or lack of network access in the builder.

Providing a pre-built image would be one way to fix it, maybe with a nice GUI progress bar when it gets downloaded on first use. Or maybe have Dangerzone build the image on first use, so it doesn't need to be hosted anywhere, but also doesn't need to be taken care of at package build time.

[yveszoundi]

I also run into this question/decision while experimenting with my own dangerzone-like tool. I chose to default to a pre-built Dockerhub image, while still allowing an option to use a custom container image (potentially self-hosted/self-built).

• Pre-installing/shipping the container as part of the package has indeed other implications/challenges (size, user feedback, etc.). I personally think that it should be an individual/conscious decision to use a self-built/hosted container image.
• I think that it's probably acceptable to allow setting or pre-setting the container image location (Dockerhub, etc.).
• Another question is when to fetch the container image (only upon installation, on startup or on-demand when the conversion starts). I much prefer the on-demand option so that the right image is pulled as needed when I update the location (dockerhub, local container engine: podman, docker, lima, etc.).

[deeplow]

Looking a bit at the project's history, I've found that pulling on install was the way it worked before. There was an image hosted on docker hub instead of it being shipped at install time.

[Rojikku]

https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=dangerzone Spent another like two hours trying to get it to build with buildah.. No dice.

This is primarily just a bump to remind people of this issue, and the fact this package hasn't worked properly for 8 months.

[apyrgio]

Thanks for the bump Rojikku. Indeed, it would be nice separating the container image from the application package. Before this happens, we need to tackle issues such as airgapped installations and privacy. But I'd like to focus on the buildah issue you have for a change, if you don't mind.

I spent some time testing Dangerzone in a fully containerized environment, and I've built the container image in one, using our provided script (./install/linux/build-image.sh). I tested out your buildah command in the same environment and it failed with:

...
STEP 3/8: RUN     wget https://gitlab.com/pdftk-java/pdftk/-/jobs/924565145/artifacts/raw/build/libs/pdftk-all.jar &&     mv pdftk-all.jar /usr/local/bin &&     chmod +x /usr/local/bin/pdftk-
all.jar &&     echo '#!/bin/sh' > /usr/local/bin/pdftk &&     echo '/usr/bin/java -jar "/usr/local/bin/pdftk-all.jar" "$@"' >> /usr/local/bin/pdftk &&     chmod +x /usr/local/bin/pdftk
Connecting to gitlab.com (172.65.251.78:443)                                                                                                                                                   wget: can't open 'pdftk-all.jar': Permission denied                                                                                                                                            
Error: building at STEP "RUN wget https://gitlab.com/pdftk-java/pdftk/-/jobs/924565145/artifacts/raw/build/libs/pdftk-all.jar &&     mv pdftk-all.jar /usr/local/bin &&     chmod +x /usr/local
/bin/pdftk-all.jar &&     echo '#!/bin/sh' > /usr/local/bin/pdftk &&     echo '/usr/bin/java -jar "/usr/local/bin/pdftk-all.jar" "$@"' >> /usr/local/bin/pdftk &&     chmod +x /usr/local/bin/p
dftk": while running runtime: exit status 1   

So, I have some questions, if you don't mind digging into this a bit:

1. What is the error that ./install/linux/build-image.sh threw in your build environment?
2. Do you see the same Buildah error as I mentioned above?
3. Does AUR have a max package size limit? If it does, we may still have an issue, even if the image builds successfully.

[Rojikku]

I think the airgap and privacy options are easily resolved by allowing people to build their own instances, just like they can now. Having one publicly built and offered wouldn't change your existing infrastructure, beyond setting one to build on say ghcr.

Building container image
time="2022-12-13T17:03:29-05:00" level=error msg="cannot find UID/GID for user builduser: no subuid ranges found for user \"builduser\" in /etc/subuid - check rootless mode in man pages."
time="2022-12-13T17:03:29-05:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
Error: creating runtime static files directory: mkdir /var/lib/containers/storage: permission denied
Saving container image
time="2022-12-13T17:03:29-05:00" level=error msg="cannot find UID/GID for user builduser: no subuid ranges found for user \"builduser\" in /etc/subuid - check rootless mode in man pages."
time="2022-12-13T17:03:29-05:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
Error: creating runtime static files directory: mkdir /var/lib/containers/storage: permission denied
Compressing container image
gzip: share/container.tar: No such file or directory
Looking up the image id
time="2022-12-13T17:03:29-05:00" level=error msg="cannot find UID/GID for user builduser: no subuid ranges found for user \"builduser\" in /etc/subuid - check rootless mode in man pages."
time="2022-12-13T17:03:29-05:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
Error: creating runtime static files directory: mkdir /var/lib/containers/storage: permission denied

These are the errors I get with build-image.sh

I would caution you with the fact that the package still builds with those errors, but does not, in fact, contain the container needed for the application to run properly. This is presumably tied to the fact that I'm running a script instead of the commands directly, and the script doesn't set an exit status when it fails.

For your second question, no.

Building container image
time="2022-12-13T17:07:09-05:00" level=warning msg="Reading allowed ID mappings: reading subuid mappings for user \"builduser\" and subgid mappings for group \"builduser\": no subuid ranges found for user \"builduser\" in /etc/subuid"
time="2022-12-13T17:07:09-05:00" level=warning msg="Found no UID ranges set aside for user \"builduser\" in /etc/subuid."
time="2022-12-13T17:07:09-05:00" level=warning msg="Found no GID ranges set aside for user \"builduser\" in /etc/subgid."
STEP 1/8: FROM alpine:latest
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/00-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715
Error: creating build container: copying system image from manifest list: writing blob: adding layer with blob "sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715": processing tar file(potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/shadow): Check /etc/subuid and /etc/subgid if configured locally and run podman-system-migrate: lchown /etc/shadow: invalid argument): exit status 1

That is the error I get.

For your third question, I feel it demonstrates a misunderstanding on how the AUR works. I'm not directly aware of any constraints, but actually the AUR doesn't host packages, the AUR hosts PKGBUILDs. More directly, it hosts the scripts to compile packages, not the packages themselves. Projects like Chaotic AUR will then build from these PKGBUILDs, and host them for people. However, it's more common for people to use "AUR helpers" that build things for them. Traditionally, the correct answer is to download the PKGBUILD, and run "makepkg -sri" and get what you want. However, doing this and maintaining updates isn't practical. So, what I use at least, is aurutils along with aurto. Essentially, a cron that checks for updates, downloads the updates pkgbuilds and compiles them if there are any, and puts those in a repo for when I next install updates. Pretty cool system, IMO. Others work similarly just more on-demand.

What these methods have in common, is that they usually chroot the pkgbuild to produce a consistent build environment without messing with the overarching environment, as well as to avoid....critical system errors in the event of a malicious maintainer, as AUR maintainers aren't considered "trusted" maintainers by default, until they build up a certain reputation.

aur build -d aurto --chroot --makepkg-conf=/etc/aurto/makepkg-chroot.conf --pacman-conf=/etc/aurto/pacman-chroot.conf PKGBUILD -f --clean Personally, I always use the above exact command to build packages when I am testing prior to pushing to the AUR. This allows me to build in a clean, reproducible environment, and avoid (some) of the common errors other maintainers run into with their users, such as missing a dependency they overlooked. You may have to adjust it depending on your setup.

Hopefully this clears things up a bit, thanks for looking into this!

[apyrgio]

Thanks a lot for the error logs @Rojikku, and the useful insights. I think I get the limitation you're seeing; the builduser does not have a subuid/subgid range in the user's system. Messing with that at build time is a no-go, so we have to think of something else. You have offered some suggestions, so we'll consider them and report back. Just note that this may take a bit longer due to this time of the year (:santa:).

[deeplow]

Hosting the image on DockerHub doesn't look like an option anymore (at least one the free plan): HN: Docker is deleting Open Source organisations - what you need to know

[AkechiShiro]

Any news on this issue ? Any new plans following @deeplow's comment ?

[apyrgio]

Unfortunately, we don't have something new in this front. We were busy moving the release infra to FPF's servers and keys, and now we're trying to tackle the Qubes OS integration, so that Dangerzone is available to SecureDrop Workstation users. These are big undertakings, and our main focus is there.

The first step towards this issue the fact that we will create .deb/.rpms for Qubes OS that will not have that container image. After that, the road will be more clear. But I don't want to make promises until we have something concrete.

Still, we may have something that will help package maintainers: we now provide the container image (for x86_64 architectures) through our release page: https://github.com/freedomofpress/dangerzone/releases/tag/v0.4.1. If build scripts can treat this image as an artifact, then hopefully we have a way forward.

@AkechiShiro, just to make sure, are you asking because you're interested in packaging Dangerzone?

[gbabiano]

Building container image
time="2022-12-13T17:07:09-05:00" level=warning msg="Reading allowed ID mappings: reading subuid mappings for user \"builduser\" and subgid mappings for group \"builduser\": no subuid ranges found for user \"builduser\" in /etc/subuid"
time="2022-12-13T17:07:09-05:00" level=warning msg="Found no UID ranges set aside for user \"builduser\" in /etc/subuid."
time="2022-12-13T17:07:09-05:00" level=warning msg="Found no GID ranges set aside for user \"builduser\" in /etc/subgid."
STEP 1/8: FROM alpine:latest
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/00-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715
Error: creating build container: copying system image from manifest list: writing blob: adding layer with blob "sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715": processing tar file(potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/shadow): Check /etc/subuid and /etc/subgid if configured locally and run podman-system-migrate: lchown /etc/shadow: invalid argument): exit status 1

That is the error I get.

@Rojikku, I found a similar error regarding the lack of sub*ids in rootless when I tried to run any of the supported Linux distributions Ubuntu / Debian / Fedora on top of Oracle Linux.

The way to overcome such issue was to include this line in Dockerfile: RUN usermod --add-subuids 10000-49999 --add-subgids 10000-49999 root

I hope this helps!