genuinetools / img

Standalone, daemon-less, unprivileged Dockerfile and OCI compatible container image builder.
https://blog.jessfraz.com/post/building-container-images-securely-on-kubernetes/
MIT License
3.89k stars 230 forks source link

nsenter: failed to unshare namespaces: Invalid argument #240

Closed dan-opitz closed 5 years ago

dan-opitz commented 5 years ago

I have seen other issues surrounding this but it sounded like a Mac problem originally so hoping to give a bit more info and see if there's a solution.

Kubernetes Server Version: v1.12.4 Container Runtime Version: docker 18.6.3

I cloned master and ran a docker build and docker push to my registry and then created a Pod spec with an initContainer to clone a repo to a volumeMount and then added the img container that referenced the cloned repo on the same volumeMount.

I am unable to set procMount: Unmasked because Unamsked is not allowed on the cluster at this time but I saw @AkihiroSuda mention this shouldn't be needed anymore.

The img container fails with the following error:

nsenter: failed to unshare namespaces: Invalid argument

Any ideas or input on how to work around/solve this issue?

Example Yaml:

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: img
    app: git-clone
  name: img
  annotations:
    container.apparmor.security.beta.kubernetes.io/img: unconfined
spec:
  securityContext:
    runAsUser: 1000
  initContainers:
    - name: git-clone
      image: docker.myregistry.com/git
      command:
        - git
        - clone
        - https://mygithub.com/myorg/myrepo.git
        - /workspace
      volumeMounts:
        - name: source
          mountPath: /workspace
  containers:
    - image: docker.myregistry.com/img
      imagePullPolicy: Always
      name: img
      resources: {}
      workingDir: /workspace
      command:
        - img
        - build
        - -t
        - testimage
        - "."
      volumeMounts:
        - name: cache-volume
          mountPath: /tmp
        - name: source
          mountPath: /workspace
  volumes:
    - name: cache-volume
      emptyDir: {}
    - name: source
      emptyDir: {}
  restartPolicy: Never
issue-label-bot[bot] commented 5 years ago

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

AkihiroSuda commented 5 years ago

container.seccomp.security.alpha.kubernetes.io/img: unconfined?

What's your node distro and kernel?

AkihiroSuda commented 5 years ago

Also make sure you are running the latest img

dan-opitz commented 5 years ago

PSP on cluster doesn't allow seccomp to be set apparently.

CentOS Linux 7 - 3.10.0-862.el7

Just tried pulling latest img and building from master with same result.

AkihiroSuda commented 5 years ago

sudo sh -c "echo 28633 > /proc/sys/user/max_user_namespaces"

ticteam commented 5 years ago

Hi, I have the same problem, if i ran your yaml example. The hint from AkihiroSuda helps if I run the docker example. But the hint from https://github.com/genuinetools/img#running-with-kubernetes , doesn't help. Or maybe you can give a little hint where to set "securityContext.privileged to true" exactly ? thx Christoph

AkihiroSuda commented 5 years ago

see https://kubernetes.io/docs/tasks/configure-pod-container/security-context/

ticteam commented 5 years ago

feedback: I was able to run the yaml without any "privileged true" but I had to comment: -# securityContext: -# allowPrivilegeEscalation: false -# readOnlyRootFilesystem: true after that the "nsenter: failed to unshare namespaces: Invalid argument" disapeared... Hope it helps ....

Asgoret commented 4 years ago

I have the same problem in OKD 3.11. Any ideas? OKD is more security paranoid than Kubernetes.

UPD#1: Run this through with jenkins master, Kubernetes plugin. Used dynamic jenkins slaves.

Asgoret commented 4 years ago

@AkihiroSuda can you help me plz?)