build fails on buildkit '#syntax' directive in Fedora guest (mount /tmp/buildkit-metadata1419535027:/run/config/buildkit/metadata (via /proc/self/fd/6), flags: 0x1021: operation not permitted) #1401

[vsiravar]

I am trying to build an image using nerdctl on Fedora guest and it stumbles on #buildkit syntax directive. The build however succeeds on Ubuntu guest.

Steps to reproduce.

[siravara@lima-fedora syntaxtests]$ cat Dockerfile
# syntax=docker/dockerfile:1
FROM public.ecr.aws/docker/library/golang:1.18 AS builder
WORKDIR /build
COPY . .

ARG TARGET_OS
ARG TARGET_ARCH
RUN go env -w GO111MODULE=off
RUN CGO_ENABLED=0 GOOS=${TARGET_OS} GOARCH=${TARGET_ARCH} go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o hello .

FROM scratch
EXPOSE 8080
COPY --from=builder /build/hello /app/
WORKDIR /app
ENTRYPOINT ["./hello"]

runc version

[siravara@lima-dfcd /]$ runc --version 
runc version 1.1.4
commit: v1.1.4-0-g5fd4c4d1
spec: 1.0.2-dev
go: go1.19.1
libseccomp: 2.5.1

BuildKit version

[siravara@lima-dfcd /]$ buildctl --version 
buildctl github.com/moby/buildkit v0.10.4 a2ba6869363812a210fcc3ded6926757ab780b5f

Logs from build

[siravara@lima-dfcd /]$ buildctl --addr=unix:///run/user/504/buildkit-default/buildkitd.sock build  --progress=auto  --frontend=dockerfile.v0 --local=context=/Users/siravara/vishwas-tests/syntaxtests/. --output=type=image,unpack=true --local=dockerfile=/Users/siravara/vishwas-tests/syntaxtests/ --opt=filename=Dockerfile
[+] Building 1.5s (4/4) FINISHED                                                
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 445B                                       0.0s
 => resolve image config for docker.io/docker/dockerfile:1                 1.3s
 => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:9ba7531bd8  0.0s
 => => resolve docker.io/docker/dockerfile:1@sha256:9ba7531bd80fb0a858632  0.0s
Dockerfile:1
--------------------
   1 | >>> # syntax=docker/dockerfile:1
   2 |     FROM public.ecr.aws/docker/library/golang:1.18 AS builder
   3 |     WORKDIR /build
--------------------
error: failed to solve: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/tmp/buildkit-metadata655966680" to rootfs at "/run/config/buildkit/metadata": mount /tmp/buildkit-metadata655966680:/run/config/buildkit/metadata (via /proc/self/fd/6), flags: 0x1021: operation not permitted: unknown

Wonder why the metadata is handled differently by BuildKit on Fedora.

[thaJeztah]

What version of runc do you have installed? I recall there were some fixes around mounts in runc v1.1.4 (not sure if relevant).

[vsiravar]

@thaJeztah We use runc version 1.1.4. Updated the issue with buildctl logs.

[vsiravar]

@thaJeztah did you get a chance to reproduce this?