rootless: permission denied (`[rootlesskit:child ] error: failed to share mount point: /: permission denied`)

[fgiannetti]

Hello all! I am trying to execute rootless buildkit inside a Kubernetes container (using Argo workflow). I followed the doc instructions but i am getting the error:

could not connect to unix:///run/user/1000/buildkit/buildkitd.sock after 10 trials ========== log ========== [rootlesskit:child ] error: failed to share mount point: /: permission denied [rootlesskit:parent] error: child exited: exit status 1 sh: can't kill pid 8: No such process

The k8s nodes are using Ubuntu 18.04.4 LTS with Kernel 4.15.0-112-generic

I cant figure out what is the problem. Is there a external configuration that I am missing?

Here is the yaml that i am using:

apiVersion: argoproj.io/v1alpha1 kind: Workflow metadata: name: buildkit-test spec: entrypoint: image templates:

• name: image inputs: artifacts:
  • name: source path: /workspace git: repo: git@github.com:myRepo.git revision: 0.0.8 sshPrivateKeySecret: name: github-creds key: ssh-private-key container: image: moby/buildkit:master-rootless workingDir: /workspace env:
    • name: BUILDKITD_FLAGS value: --oci-worker-no-process-sandbox command:
    • buildctl-daemonless.sh args:
    • build
    • --frontend
    • dockerfile.v0
    • --local
    • context=.
    • --local
    • dockerfile=.
    • --output
    • type=image,name=hub.docker/myRepo:0.0.8-BK,push=true

Thanks very much!

[fgiannetti]

Additional info: Running the same yaml in a local minikube cluster, it works fine!

[AkihiroSuda]

https://github.com/moby/buildkit/blob/0279989d7f0df0fc98a4f32b4c6650dde2a42107/examples/buildctl-daemonless/buildctl-daemonless.sh#L15

Please try changing rootlesskit to rootlesskit --propagation=rslave

[fgiannetti]

Hello Akiro! Thanks for your answer!

I get the same error, but with an aditional warning message: "The host root filesystem is mounted as \"master:700\". Setting child propagation to \"rslave\" is not supported."

Is a Warning but could this be the reason for the error?

Thanks again!

[unckleg]

Same error's here. Running on GKE.

[unckleg]

@fgiannetti

I've got it working with:

securityContext:
  privileged: true

So try adding it to your task:

  args:
    - build
    - --frontend
    - dockerfile.v0
    - --local
    - context=.
    - --local
    - dockerfile=.
    - --output
    - type=image,name=hub.docker/myRepo:0.0.8-BK,push=true
  securityContext:
    privileged: true

[fgiannetti]

@unckleg Adding that config now it is working! Thank you so much!

[leeweisberger]

I am running into the same error running on GKE (ubuntu based image, both containerd and docker) with moby/buildkit:v0.10.0-rootless and buildctl-daemonless.sh. I don't want to add privileged: true to my pod and the instructions claim that this should work without privileged.

Any other thoughts here? Can we please reopen this issue? This last hurdle is blocking adoption of BuildKit for us!

[unckleg]

@leeweisberger What executor are you using on top of BuildKit? Please share a wider context 👀

[leeweisberger]

For anyone else who gets this. make sure you disable apparmour. Once I did that, the issue went away!

[AbimbolaE]

Instead of disabling apparmour entirely as @leeweisberger suggested, a more secure alternative would be to add the unconfined apparmour profile to your buildkitd container

For example ...

spec:
  replicas: 1
  selector:
    matchLabels:
      app: buildkitd
  template:
    metadata:
      labels:
        app: buildkitd
      annotations:
        container.apparmor.security.beta.kubernetes.io/buildkitd: unconfined  # <-- here

References

• Kubernetes Apparmour - Securing a Pod: https://kubernetes.io/docs/tutorials/security/apparmor/#securing-a-pod
• Moby BuildKitd - Example of Kubernetes Service and Deployment: https://github.com/moby/buildkit/blob/master/examples/kubernetes/deployment%2Bservice.rootless.yaml