Building without --privileged under an alpine container in Docker

[beaugunderson]

yo, thank you for img! one thing i'm trying to clarify--is it possible to build inside an alpine docker container without using --privileged? the README says this:

Docker image r.j3ss.co/img is configured to be executed as an unprivileged user with UID 1000 and it does not need --privileged since img v0.6.0.

but there's no current release for v0.6.0, and it looks like it was reverted right after it was released...

[issue-label-bot[bot]]

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

[AkihiroSuda]

What do you mean by reverted?

[beaugunderson]

0.6.0 was reverted in this commit, yet the README refers to it as the version that works without --privileged

[AkihiroSuda]

the reverted commit seems very old and unrelated.

The planned 0.6.0 will support running without --privileged but not released yet. (You can try master)

[AkihiroSuda]

@jessfraz could you release 0.6.0?

[AndyBitz]

I've just tried it with master and got this error:

failed to solve: failed to read dockerfile: failed to mount /tmp/buildkit-mount404401060: [{Type:bind Source:/tmp/.local/share/img/runc/native/snapshots/snapshots/2 Options:[rbind ro]}]: operation not permitted

@AkihiroSuda are you sure it will be supported?

[AkihiroSuda]

are you using gcos? https://github.com/moby/buildkit/issues/879

[AndyBitz]

@AkihiroSuda No, I've tried it locally on my MacBook with docker run. It only works when I use --privileged

[AkihiroSuda]

Note that you still need seccomp=unconfined and apparmor=unconfined

[AndyBitz]

I've tried it with those too, but only the --privileged makes a difference

[AkihiroSuda]

Whats your kernel?

[AndyBitz]

4.9.125-linuxkit

[AndyBitz]

@AkihiroSuda my goal is to run this on the amazonlinux base image.

[AkihiroSuda]

it works for me (4.9.125-linuxkit, Docker 18.09.3, img@013bb211)

$ docker run --rm -it --security-opt seccomp=unconfined --security-opt apparmor=unconfined -v $(pwd):/home/user/src r.j3ss.co/img build -t foo /home/user/src
Building docker.io/library/foo:latest
Setting up the rootfs... this may take a bit.
WARN[0000] Process sandbox is not available, consider unmasking procfs: mount: permission denied (are you root?)

[+] Building 23.0s (6/6) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                0.1s
 => => transferring dockerfile: 76B                                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                                   0.1s
 => => transferring context: 2B                                                                                                                                                     0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                                                                                                    3.3s
 => [1/2] FROM docker.io/library/alpine@sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913                                                                     2.2s
 => => resolve docker.io/library/alpine@sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913                                                                     0.0s
 => => sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913 2.36kB / 2.36kB                                                                                      0.0s
 => => sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 528B / 528B                                                                                          0.0s
 => => sha256:bdf0201b3a056acc4d6062cc88cd8a4ad5979983bfb640f15a145e09ed985f92 2.76MB / 2.76MB                                                                                      1.8s
 => => sha256:cdf98d1859c1beb33ec70507249d34bacf888d59c24df3204057f9a6c758dddb 1.51kB / 1.51kB                                                                                      0.0s
 => => unpacking docker.io/library/alpine@sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913                                                                   0.3s
 => [2/2] RUN apk add --no-cache gcc                                                                                                                                                5.5s
 => exporting to image                                                                                                                                                             11.8s
 => => exporting layers                                                                                                                                                            11.7s
 => => exporting manifest sha256:184a1f8e1649c7f9276e1130d61ff4db8536239e0f4e408b9632c039d02dd3d6                                                                                   0.0s
 => => exporting config sha256:73863d87f4822166bd36bd32a6f8812439c89c07157d9a558b4be218f16b662a                                                                                     0.0s
 => => naming to docker.io/library/foo:latest                                                                                                                                       0.0s
Successfully built docker.io/library/foo:latest

[rodriguezsergio]

I can't seem to get this working with only those 2 security-opt options as well. I'm running an jenkins/jnlp-slave:latest-jdk11 image with the precompiled img 0.5.7 release installed. Running with --privileged resolves it.

[jankais3r]

Apologies for hijacking the issue, but I am getting the same error as Andy:

Jans-iPhone:~# img build -d -t hello .
DEBU[0000] checking runc                                 disableEmbeddedRunc=false state=/root/.local/share/img
DEBU[0000] runc found                                    commit=56aca5aa50d07548d5db8fd33e9dc562f70f3208 spec=1.0.2 version=1.0.0-rc10+dev
DEBU[0000] using backend: native                        
Building docker.io/library/hello:latest
Setting up the rootfs... this may take a bit.
DEBU[0000] serving grpc connection                      
WARN[0000] Process sandbox is not available, consider unmasking procfs:  
WARN[0000] using host network as the default            
[+] Building 0.0s (0/0)                           
DEBU[0000] reusing ref for local: vt0z8kf22gjwit9y6wqt9s69y 
DEBU[0000] reusing ref for local: lnql54ycndslsamk[+] Building 0.0s (0/2)                           
 => [internal] load build definition from D  0.0s
 => => transferring dockerfile:              0.0s
 => [internal] load .dockerignore            0.0s
 => => transferring context:                 0.0s
DEBU[0000] diffcopy took: 57.044ms                      
DEBU[0000] saved vt0z8kf22gjwit9y6wqt9s69y as loca[+] Building 0.1s (2/2) FINISHED                  
 => [internal] load build definition from D  0.1s 
 => => transferring dockerfile: 2B           0.1s
 => [internal] load .dockerignore            0.1sa
 => => transferring context: 2B              0.1s
Error: failed to solve: failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to mount /tmp/buildkit-mount078545067: [{Type:bind Source:/root/.local/share/img/runc/native/snapshots/snapshots/1 Options:[rbind ro]}]: invalid argument

The problem is that I am not running this inside a Docker container, but directly on Alpine Linux 3.2.0 i686 kernel. Therefore I cannot just add --privileged. Any advice how to avoid this error when running directly on Alpine? I cannot use Docker.

[bbkgh]

We have this problem too. Does anyone have some workaround about this? (to build with img in alpine and unprivileged)

[AkihiroSuda]

The BuildKit upstream is more actively maintained and has better documentation about running without --privileged https://github.com/moby/buildkit/blob/master/docs/rootless.md#docker

[baszalmstra]

Im running into the same issue when running Gitlab CI with the kubernetes executor. The build job uses an alpine image which fails with:

Error: failed to solve: failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to mount /tmp/buildkit-mount078545067: [{Type:bind Source:/root/.local/share/img/runc/native/snapshots/snapshots/1 Options:[rbind ro]}]: invalid argument

Adding the pod annotations as suggested in the readme didn't have any effect:

[runners.kubernetes.pod_annotations]
  "container.apparmor.security.beta.kubernetes.io/build" = "unconfined"
  "container.seccomp.security.alpha.kubernetes.io/build" = "unconfined"

Only privileged mode seems to fix this.

[oxr463]

Im running into the same issue when running Gitlab CI with the kubernetes executor. The build job uses an alpine image which fails with:

Error: failed to solve: failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to mount /tmp/buildkit-mount078545067: [{Type:bind Source:/root/.local/share/img/runc/native/snapshots/snapshots/1 Options:[rbind ro]}]: invalid argument

Adding the pod annotations as suggested in the readme didn't have any effect:

[runners.kubernetes.pod_annotations]
  "container.apparmor.security.beta.kubernetes.io/build" = "unconfined"
  "container.seccomp.security.alpha.kubernetes.io/build" = "unconfined"

Only privileged mode seems to fix this.

Are you building as root?

[baszalmstra]

I dont exactly know what you mean. Im basically running alpine:3.13, I add img with apk add img and run img build:

The gitlab ci job looks like this:

docker:
  image: alpine:3.13
  stage: test
  script:
    - apk add --update --no-cache --quiet img
    - img build --tag ${CI_REGISTRY_IMAGE} .

[AkihiroSuda]

If you are using Google COS kernel, probably you are hitting this https://github.com/moby/buildkit/issues/879

[baszalmstra]

Im running all of this on a DigitalOcean Managed Kubernetes cluster I don't think they are using the same kernel.

[MrAmbiG]

No resolution yet ?