catthehacker / docker_images

Docker images
MIT License
212 stars 75 forks source link

Trouble mounting `binfmt_misc` for Docker builds #112

Closed lstellway closed 1 month ago

lstellway commented 11 months ago

Hello,

I'm currently using Gitea with the Gitea act runner. I have the following labels configured to run jobs:

  labels:
    - "ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-latest"
    - "ubuntu-22.04:docker://ghcr.io/catthehacker/ubuntu:act-22.04"
    - "ubuntu-20.04:docker://ghcr.io/catthehacker/ubuntu:act-20.04"

When trying to use the docker/setup-qemu-action@v3 action to prepare the environment for cross-platform builds, I get the following error:

[command]/usr/bin/docker run --rm --privileged tonistiigi/binfmt:latest --install all
error: operation not permitted
cannot mount binfmt_misc filesystem at /proc/sys/fs/binfmt_misc

main.run
    /src/cmd/binfmt/main.go:183
main.main
    /src/cmd/binfmt/main.go:170
runtime.main
    /usr/local/go/src/runtime/proc.go:250
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1571

Anybody else have this issue? Any ideas how to get the mount working?

I'm thinking it maybe has something to do with the Docker daemon running outside of this container's filesystem and not having permissions to write to the mount path..?

Here is a sample of my workflow:

name: release-tag

on:
  push:
    tags: ["*"]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
lstellway commented 11 months ago

Additional notes ...

QEMU script

I came across this script from the qemu/qemu repo (via this article) for setting up QEMU. The script has an option specific to setting up QEMU on Debian (Ubuntu is based on Debian).

When executing this script within the container environment, I need to additionally install an extra package: binfmt-support. Afterwards, I can successfully mount QEMU with the --debian option.

Sample workflow step:

      - name: Set up QEMU
        run: |
          sudo apt-get update
          sudo apt-get install -y binfmt-support
          curl -L -o /tmp/qemu.sh 'https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-binfmt-conf.sh'
          chmod +x /tmp/qemu.sh
          /tmp/qemu.sh --debian

With this, my /usr/share/binfmts/ directory is populated with the following:

ls -l /usr/share/binfmts/ ```txt python3.10 qemu-sh4 qemu-microblaze qemu-alpha qemu-loongarch64 qemu-armeb qemu-sparc32plus qemu-microblazeel qemu-aarch64_be qemu-ppc qemu-mipsn32 qemu-riscv64 qemu-mipsel qemu-riscv32 qemu-sparc64 qemu-mipsn32el qemu-ppc64 qemu-xtensa qemu-aarch64 qemu-hppa qemu-arm qemu-mips64el qemu-mips qemu-ppc64le qemu-or1k qemu-hexagon qemu-s390x qemu-m68k qemu-xtensaeb qemu-mips64 qemu-sparc qemu-sh4eb ```

But the only platforms available to Docker buildx are:

linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/386

Run via Docker

As documented in multiarch/qemu-user-static, this script can also be run via Docker:

docker run --rm --privileged multiarch/qemu-user-static:register [--reset][--help][options]

My Gitea-runner host is in an Alpine linux container _(gitea/act_runner:0.2.6-dind-rootless)_.

When I run this with no options:

docker run --rm --privileged multiarch/qemu-user-static:register

I get the error:

mount: permission denied (are you root?)

When I run it with the --debian flag:

docker run --rm --privileged multiarch/qemu-user-static:register --debian

I get the same error with a warning (source reference):

mount: permission denied (are you root?) WARNING: your system is not a Debian based distro

Related issue

Also found an issue in the action's repository that looks related: docker/setup-qemu-action#67

catthehacker commented 1 month ago

You are trying to run a root action in rootless daemon. No idea how to help.