Feature Request: rootless build with podman

wfrisch commented 1 year ago

Podman is a container engine that supports rootless containers.

obs-build could leverage this to allow for building packages without root privileges. Conveniently there's also Docker emulation available (zypper in podman-docker), so you could reuse the existing Docker support.

In fact it partially works already:

OSC_SU_WRAPPER= osc build --vm-type=docker --root=$HOME/build-root

→ A rootless podman container is created and runs for a while...

[    0s] booting docker...
[    0s] Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
[    1s] 2nd stage started in virtual machine
[    1s] machine type: x86_64
[    1s] Linux version: 6.1.7-1-default #1 SMP PREEMPT_DYNAMIC Wed Jan 18 11:12:34 UTC 2023 (872045c)
[    1s] Increasing log level from now on...
[    1s] Enable sysrq operations

Some bits are still failing, e.g. the creation of device nodes:

[    2s] mknod: //dev/null: Operation not permitted

What do you think?

fcrozat commented 1 year ago

init_buildsystem should not try to create devices if they are already present thanks to container engine.

Additionally, it would be nice to support vm-type=podman so we don't need to install podman-docker.

Vogtinator commented 12 months ago

FWICT OSC_SU_WRAPPER= osc build --vm-type=podman does work meanwhile so this can be closed?

osc just needs to learn that --vm-type=podman does not need elevated privs.

Vogtinator commented 12 months ago

osc just needs to learn that --vm-type=podman does not need elevated privs.

Filed as https://github.com/openSUSE/osc/issues/1410

mcepl commented 11 months ago

On Mon Sep 18, 2023 at 2:15 PM CEST, Fabian Vogt wrote:

osc just needs to learn that --vm-type=podman does not need elevated privs.

Filed as https://github.com/openSUSE/osc/issues/1410

How does this relate to the similar problem of running osc build from inside of a podman container (e.g., from distrobox container on MicroOS-base system)?

Vogtinator commented 11 months ago

On Mon Sep 18, 2023 at 2:15 PM CEST, Fabian Vogt wrote: > osc just needs to learn that --vm-type=podman does not need elevated privs. Filed as openSUSE/osc#1410 How does this relate to the similar problem of running osc build from inside of a podman container (e.g., from distrobox container on MicroOS-base system)?

I haven't tried. In theory if everything is set up to allow running nested podman, it might just work.

fcrozat commented 11 months ago

it is usually easier to just have a symlink to distrobox-host-exec to /usr/local/bin/podman in the distrobox container ;)

wfrisch commented 11 months ago

Solved by https://github.com/openSUSE/osc/pull/1415 Thank you all! I'm really enjoying this new feature.

mcepl commented 11 months ago

I probably do something wrong, but in non-root container (at 19d1bab7) I get this:

tumbleweed-pkg~/b/n/vis (master)$ oscb --vm-type=podman --debug
hostarch:  x86_64
arg_arch:  x86_64
arg_repository:  openSUSE_Tumbleweed
arg_descr:  None
DEBUG: makeurl: https://api.opensuse.org ['source', 'home:mcepl:neovim', '_meta'] []
Building vis.spec for openSUSE_Tumbleweed/x86_64
DEBUG: makeurl: https://api.opensuse.org ['source', 'home:mcepl:neovim', 'vis'] cmd=getprojectservices
Getting buildconfig from server and store to /home/matej/build/neovim/vis/.osc/_buildconfig-openSUSE_Tumbleweed-x86_64
DEBUG: makeurl: https://api.opensuse.org ['build', 'home:mcepl:neovim', 'openSUSE_Tumbleweed', '_buildconfig'] []
Getting buildinfo from server and store to /home/matej/build/neovim/vis/.osc/_buildinfo-openSUSE_Tumbleweed-x86_64.xml
DEBUG: makeurl: https://api.opensuse.org ['build', 'home:mcepl:neovim', 'openSUSE_Tumbleweed', 'x86_64', 'vis', '_buildinfo'] ['add=gdb', 'add=vim', 'add=gdb', 'add=vim', 'add=strace', 'add=less']
Updating cache of required packages
0.0% cache miss. 244/244 dependencies cached.

DEBUG: makeurl: https://api.opensuse.org ['source', 'openSUSE:Tumbleweed', '_pubkey'] []
DEBUG: makeurl: https://api.opensuse.org ['source', 'editors', '_pubkey'] []
Verifying integrity of cached packages
using keys from openSUSE:Tumbleweed, editors
Writing build configuration
Running build
logging output to /var/tmp/build-root-matej/openSUSE_Tumbleweed-x86_64/.build.log...
[    0s] Memory limit set to 10059072KB
[    0s] Using BUILD_ROOT=/var/tmp/build-root-matej/openSUSE_Tumbleweed-x86_64/.mount
[    0s] Using BUILD_ARCH=x86_64:i686:i586:i486:i386
[    0s] Doing podman build
[    0s] 
tumbleweed-pkg~/b/n/vis (master)$ oscb --vm-type=podman --debug
hostarch:  x86_64
arg_arch:  x86_64
arg_repository:  openSUSE_Tumbleweed
arg_descr:  None
DEBUG: makeurl: https://api.opensuse.org ['source', 'home:mcepl:neovim', '_meta'] []
Building vis.spec for openSUSE_Tumbleweed/x86_64
DEBUG: makeurl: https://api.opensuse.org ['source', 'home:mcepl:neovim', 'vis'] cmd=getprojectservices
Getting buildconfig from server and store to /home/matej/build/neovim/vis/.osc/_buildconfig-openSUSE_Tumbleweed-x86_64
DEBUG: makeurl: https://api.opensuse.org ['build', 'home:mcepl:neovim', 'openSUSE_Tumbleweed', '_buildconfig'] []
Getting buildinfo from server and store to /home/matej/build/neovim/vis/.osc/_buildinfo-openSUSE_Tumbleweed-x86_64.xml
DEBUG: makeurl: https://api.opensuse.org ['build', 'home:mcepl:neovim', 'openSUSE_Tumbleweed', 'x86_64', 'vis', '_buildinfo'] ['add=gdb', 'add=vim', 'add=gdb', 'add=vim', 'add=strace', 'add=less']
Updating cache of required packages
0.0% cache miss. 244/244 dependencies cached.

DEBUG: makeurl: https://api.opensuse.org ['source', 'openSUSE:Tumbleweed', '_pubkey'] []
DEBUG: makeurl: https://api.opensuse.org ['source', 'editors', '_pubkey'] []
Verifying integrity of cached packages
using keys from openSUSE:Tumbleweed, editors
Writing build configuration
Running build
logging output to /var/tmp/build-root-matej/openSUSE_Tumbleweed-x86_64/.build.log...
[    0s] Memory limit set to 10059072KB
[    0s] Using BUILD_ROOT=/var/tmp/build-root-matej/openSUSE_Tumbleweed-x86_64/.mount
[    0s] Using BUILD_ARCH=x86_64:i686:i586:i486:i386
[    0s] Doing podman build
[    0s] 
[    0s] 
[    0s] tumbleweed-pkg.mitmanek.cepl.eu started "build vis.spec" at Mon Oct  9 23:56:24 UTC 2023.
[    0s] 
[    0s] 
[    0s] processing recipe /home/matej/build/neovim/vis/vis.spec ...
[    0s] running changelog2spec --target rpm --file /home/matej/build/neovim/vis/vis.spec
[    0s] init_buildsystem --configdir /usr/lib/build/configs --cachedir /home/matej/.cache/opensuse.org/build/cache --prepare --rpmlist /tmp/rpmlist.h19_lm9j /home/matej/build/neovim/vis/vis.spec ...
[    0s] copying packages...
[    0s] reordering...cycle: libncurses6 -> terminfo-base
[    0s]   breaking dependency terminfo-base -> libncurses6
[    0s] cycle: binutils -> libctf0
[    0s]   breaking dependency binutils -> libctf0
[    0s] cycle: openssl -> openssl-3
[    0s]   breaking dependency openssl-3 -> openssl
[    0s] cycle: python311-base -> libpython3_11-1_0
[    0s]   breaking dependency python311-base -> libpython3_11-1_0
[    0s] cycle: python311-Sphinx -> python311-sphinx_rtd_theme
[    0s]   breaking dependency python311-Sphinx -> python311-sphinx_rtd_theme
[    0s] cycle: python311-sphinxcontrib-applehelp -> python311-Sphinx
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-applehelp
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-devhelp
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-devhelp
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-htmlhelp
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-htmlhelp
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-jsmath
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-jsmath
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-qthelp
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-qthelp
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-serializinghtml
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-serializinghtml
[    0s] cycle: rpm-config-SUSE -> rpm
[    0s]   breaking dependency rpm -> rpm-config-SUSE
[    0s] done
[    1s] booting podman...
[    1s] Error: statfs /var/tmp/build-root-matej/openSUSE_Tumbleweed-x86_64/.mount: no such file or directory

Build failed with exit code 125
The buildroot was: /var/tmp/build-root-matej/openSUSE_Tumbleweed-x86_64/.mount

Cleaning the build root may fix the problem or allow you to start debugging from a well-defined state:
  - add '--clean' option to your 'osc build' command
  - run 'osc wipe [--vm-type=...]' prior running your 'osc build' command again
tumbleweed-pkg~/b/n/vis (master)$ l /usr/local/bin/podman 
lrwxrwxrwx. 1 root root 29 říj 10 01:51 /usr/local/bin/podman -> ../../bin/distrobox-host-exec
tumbleweed-pkg~/b/n/vis (master)$

What am I missing?

mcepl commented 11 months ago

When I remove /usr/local/bin/podman link and install full package, I get:

$ oscb --vm-type=podman --debug
hostarch:  x86_64
arg_arch:  x86_64
arg_repository:  openSUSE_Tumbleweed
arg_descr:  None
DEBUG: makeurl: https://api.opensuse.org ['source', 'home:mcepl:neovim', '_meta'] []
Building vis.spec for openSUSE_Tumbleweed/x86_64
DEBUG: makeurl: https://api.opensuse.org ['source', 'home:mcepl:neovim', 'vis'] cmd=getprojectservices
Getting buildconfig from server and store to /home/matej/build/neovim/vis/.osc/_buildconfig-openSUSE_Tumbleweed-x86_64
DEBUG: makeurl: https://api.opensuse.org ['build', 'home:mcepl:neovim', 'openSUSE_Tumbleweed', '_buildconfig'] []
Getting buildinfo from server and store to /home/matej/build/neovim/vis/.osc/_buildinfo-openSUSE_Tumbleweed-x86_64.xml
DEBUG: makeurl: https://api.opensuse.org ['build', 'home:mcepl:neovim', 'openSUSE_Tumbleweed', 'x86_64', 'vis', '_buildinfo'] ['add=gdb', 'add=vim', 'add=gdb', 'add=vim', 'add=strace', 'add=less']
Updating cache of required packages
0.0% cache miss. 244/244 dependencies cached.

DEBUG: makeurl: https://api.opensuse.org ['source', 'openSUSE:Tumbleweed', '_pubkey'] []
DEBUG: makeurl: https://api.opensuse.org ['source', 'editors', '_pubkey'] []
Verifying integrity of cached packages
using keys from openSUSE:Tumbleweed, editors
Writing build configuration
Running build
logging output to /var/tmp/build-root-matej/openSUSE_Tumbleweed-x86_64/.build.log...
[    0s] Memory limit set to 10059072KB
[    0s] Using BUILD_ROOT=/var/tmp/build-root-matej/openSUSE_Tumbleweed-x86_64/.mount
[    0s] Using BUILD_ARCH=x86_64:i686:i586:i486:i386
[    0s] Doing podman build
[    0s] 
[    0s] 
[    0s] tumbleweed-pkg.mitmanek.cepl.eu started "build vis.spec" at Tue Oct 10 00:01:31 UTC 2023.
[    0s] 
[    0s] 
[    0s] processing recipe /home/matej/build/neovim/vis/vis.spec ...
[    0s] running changelog2spec --target rpm --file /home/matej/build/neovim/vis/vis.spec
[    0s] init_buildsystem --configdir /usr/lib/build/configs --cachedir /home/matej/.cache/opensuse.org/build/cache --prepare --rpmlist /tmp/rpmlist.0in6hbaq /home/matej/build/neovim/vis/vis.spec ...
[    0s] copying packages...
[    0s] reordering...cycle: libncurses6 -> terminfo-base
[    0s]   breaking dependency terminfo-base -> libncurses6
[    0s] cycle: binutils -> libctf0
[    0s]   breaking dependency binutils -> libctf0
[    0s] cycle: openssl -> openssl-3
[    0s]   breaking dependency openssl-3 -> openssl
[    0s] cycle: python311-base -> libpython3_11-1_0
[    0s]   breaking dependency python311-base -> libpython3_11-1_0
[    0s] cycle: python311-Sphinx -> python311-sphinx_rtd_theme
[    0s]   breaking dependency python311-Sphinx -> python311-sphinx_rtd_theme
[    0s] cycle: python311-sphinxcontrib-applehelp -> python311-Sphinx
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-applehelp
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-devhelp
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-devhelp
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-htmlhelp
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-htmlhelp
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-jsmath
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-jsmath
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-qthelp
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-qthelp
[    0s] cycle: python311-Sphinx -> python311-sphinxcontrib-serializinghtml
[    0s]   breaking dependency python311-Sphinx -> python311-sphinxcontrib-serializinghtml
[    0s] cycle: rpm-config-SUSE -> rpm
[    0s]   breaking dependency rpm -> rpm-config-SUSE
[    0s] done
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"btrfs\" from database - delete libpod local files (\"/home/matej/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="cannot find UID/GID for user matej: no subuid ranges found for user \"matej\" in /etc/subuid - check rootless mode in man pages."
[    1s] time="2023-10-10T02:01:32+02:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="cannot find UID/GID for user matej: no subuid ranges found for user \"matej\" in /etc/subuid - check rootless mode in man pages."
[    1s] time="2023-10-10T02:01:32+02:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="cannot find UID/GID for user matej: no subuid ranges found for user \"matej\" in /etc/subuid - check rootless mode in man pages."
[    1s] time="2023-10-10T02:01:32+02:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="invalid internal status, try resetting the pause process with \"podman system migrate\": cannot re-exec process to join the existing user namespace"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"btrfs\" from database - delete libpod local files (\"/home/matej/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="cannot find UID/GID for user matej: no subuid ranges found for user \"matej\" in /etc/subuid - check rootless mode in man pages."
[    1s] time="2023-10-10T02:01:32+02:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="cannot find UID/GID for user matej: no subuid ranges found for user \"matej\" in /etc/subuid - check rootless mode in man pages."
[    1s] time="2023-10-10T02:01:32+02:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="cannot find UID/GID for user matej: no subuid ranges found for user \"matej\" in /etc/subuid - check rootless mode in man pages."
[    1s] time="2023-10-10T02:01:32+02:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="invalid internal status, try resetting the pause process with \"podman system migrate\": cannot re-exec process to join the existing user namespace"
[    1s] booting podman...
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"btrfs\" from database - delete libpod local files (\"/home/matej/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="cannot find UID/GID for user matej: no subuid ranges found for user \"matej\" in /etc/subuid - check rootless mode in man pages."
[    1s] time="2023-10-10T02:01:32+02:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="cannot find UID/GID for user matej: no subuid ranges found for user \"matej\" in /etc/subuid - check rootless mode in man pages."
[    1s] time="2023-10-10T02:01:32+02:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="cannot find UID/GID for user matej: no subuid ranges found for user \"matej\" in /etc/subuid - check rootless mode in man pages."
[    1s] time="2023-10-10T02:01:32+02:00" level=warning msg="Using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids if not using a network user"
[    1s] time="2023-10-10T02:01:32+02:00" level=error msg="invalid internal status, try resetting the pause process with \"podman system migrate\": cannot re-exec process to join the existing user namespace"
[    1s] 
[    1s] tumbleweed-pkg.mitmanek.cepl.eu failed "build vis.spec" at Tue Oct 10 00:01:32 UTC 2023.
[    1s] 

Build failed with exit code 1
The buildroot was: /var/tmp/build-root-matej/openSUSE_Tumbleweed-x86_64/.mount

Cleaning the build root may fix the problem or allow you to start debugging from a well-defined state:
  - add '--clean' option to your 'osc build' command
  - run 'osc wipe [--vm-type=...]' prior running your 'osc build' command again
$

wfrisch commented 10 months ago

I'm also having trouble with the current version in openSUSE:Factory:

cd openSUSE:Factory/moreutils
osc build --vm-type=podman

[    0s] running changelog2spec --target rpm --file /home/test/openSUSE:Factory/moreutils/moreutils.spec
[    0s] init_buildsystem --configdir /usr/lib/build/configs --cachedir /home/test/.cache/opensuse.org/build/cache --prepare --rpmlist /tmp/rpmlist.anwii4fy /home/test/openSUSE:Factory/moreutils/moreutils.spec ...
[    0s] unpacking preinstall image openSUSE:Factory/standard/preinstallimage-base [193ea1172dcbf8835a9dbbe82e9e54ba]
[    0s] bsdtar: Can't chroot to ".": Operation not permitted
[    0s] ERROR: unpack failed.
[    0s] ERROR: This is a .zst compressed preinstallimage and /usr/bin/bsdtar --exclude .build --exclude .init_b_cache -P --chroot --numeric-owner -x failed to unpack.
[    0s] Try to enable zst in /usr/bin/bsdtar --exclude .build --exclude .init_b_cache -P --chroot --numeric-owner -x or disable preinstallimage.

With --nopreinstallimage:

cd openSUSE:Factory/moreutils
osc build --vm-type=podman --nopreinstallimage

[    3s] booting podman...
[    4s] Error: OCI runtime error: crun: mount `/var/tmp/build-root-test/standard-x86_64/.mount` to ``: Invalid argument

Vogtinator commented 10 months ago

The preinstallimage part should be fixed by osc #1444 already.

The podman error could be a regression by the switch to crun?

Vogtinator commented 10 months ago

The podman error could be a regression by the switch to crun?

Yes, confirmed. It breaks after zypper in crun and works after removing it again.

wfrisch commented 10 months ago

The podman error could be a regression by the switch to crun?

Yes, confirmed. It breaks after zypper in crun and works after removing it again.

Excellent, thanks! Unfortunately podman in Factory depends on crun (for WASM support only). As a temporary workaround rm /usr/bin/crun works for me.

dirkmueller commented 9 months ago

A better workaround is to set the runtime back to runc in /etc/containers/containers.conf

I'm looking at resolving there issue with crun or maybe converting the default back.

openSUSE / obs-build

Feature Request: rootless build with podman #914