search

chainguard-dev / melange

build APKs from source code
Apache License 2.0
427 stars 110 forks source link

Problems running on Ubuntu with apparmor / restricted unprivileged userns #1508

Open smoser opened 1 month ago

smoser commented 1 month ago

Problem

Running melange for the first time on a Ubuntu system will often fail. There are a couple hangups.

  1. need to install bubblewrap (apt-get install bubblewrap). This actually gives a pretty good error:

    2024/09/19 10:33:22 WARN SOURCE_DATE_EPOCH is specified but empty, setting it to 1969-12-31 19:00:00 -0500 EST
2024/09/19 10:33:22 WARN cannot use bubblewrap for containers: bwrap not found on $PATH arch=x86_64
2024/09/19 10:33:22 ERRO unable to run containers using bubblewrap, specify --runner and one of [bubblewrap docker qemu]

  2. Current ubuntu systems require application specific configs in order to use unprivileged user namespaces

    The failure path in wolfi-dev/os looks like this:

    $ make package/mypkg
@SOURCE_DATE_EPOCH= /home/smoser/go/bin/melange build mypkg.yaml \
   --repository-append /home/smoser/src/wolfi/os/packages --keyring-append local-melange.rsa.pub\
   --signing-key local-melange.rsa --arch x86_64 --env-file build-x86_64.env \
  --namespace wolfi --generate-index false  --pipeline-dir ./pipelines/ \
  -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub \
  -r https://packages.wolfi.dev/os
...
2024/09/19 10:33:58 INFO built image layer tarball as /tmp/apko-temp-1864610170/apko-x86_64.tar.gz
2024/09/19 10:33:58 INFO using /tmp/apko-temp-1864610170/apko-x86_64.tar.gz for image layer
2024/09/19 10:33:58 INFO ImgRef = /tmp/melange-guest-2642779980
2024/09/19 10:33:58 WARN bwrap: setting up uid map: Permission denied
2024/09/19 10:33:58 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/09/19 10:33:58 INFO   workspace dir: /tmp/melange-workspace-4104388352
2024/09/19 10:33:58 INFO   guest dir: /tmp/melange-guest-1722197694
2024/09/19 10:33:58 ERRO failed to build package: unable to start pod: exit status 1

The unprivileged user namespace path is a bit of a rathole. Here is some reading if you're interested:

How to fix

There are at least the following options to fix

  1. disable the apparmor userns restrictions Add kernel.apparmor_restrict_unprivileged_userns = 0 to a file in /etc/sysctl.d/60-apparmor-namespace.conf (per apparmor doc)

  2. Allow bubblewrap to use unprivileged user namespaces independent of who calls it.

    Create the file /etc/apparmor.d/local-bwrap with content like below:

    abi <abi/4.0>,
include <tunables/global>

profile local-bwrap /usr/bin/bwrap flags=(unconfined) {
  userns,

  # Site-specific additions and overrides. See local/README for details.
  include if exists <local/bwrap>
}

    Then run sudo systemctl reload apparmor

  3. Allow melange to use unprivileged user namespaces.

    Create the file /etc/apparmor.d/local-melange with content like below (update '/path/to/your/home/...')

    abi <abi/4.0>,
include <tunables/global>

profile local-melange /path/to/your/home/go/bin/melange flags=(unconfined) {
  userns,

  # Site-specific additions and overrides. See local/README for details.
  include if exists <local/melange>
}

    Then run sudo systemctl reload apparmor

smoser commented 1 month ago

I wanted to dump the above from my head, and would have put a PR up with the content, but didn't know where it should go. Let me know where it should go and I will do so.

Thanks @murraybd for pushing on this a bit.

murraybd commented 1 month ago

I never restarted apparmor rather I used sudo service apparmor reload.

smoser commented 1 month ago

I never restarted apparmor rather I used sudo service apparmor reload.

updated. thank you.

justinvreeland commented 1 month ago

AppArmor can't be reloaded so the service helper scripts run restart. Or at least they do on Debian. If you're using systemd it should be restart. If you're using the helper scripts reload will restart.