search

genuinetools / img

Standalone, daemon-less, unprivileged Dockerfile and OCI compatible container image builder.
https://blog.jessfraz.com/post/building-container-images-securely-on-kubernetes/
MIT License
3.88k stars 229 forks source link

img on kubernetes with allowPrivilegeEscalation: false #355

Closed maigl closed 1 year ago

maigl commented 2 years ago

Hi,

I was able to run img successfully. But now my company private-cloud kubernetes cluster provider changed the PSP and enforces:

   securityContext:
      allowPrivilegeEscalation: false
      privileged: false

The main reason seems to be to not allow sudo and other dangerous things inside the container.. And now img does not work anymore and gives me the following error:

$ img build . -t test  
newuidmap: Could not set caps
nsenter: failed to use newuidmap: Invalid argument
nsenter: failed to sync with parent: SYNC_USERMAP_ACK: got 255: Invalid argument

Do I have any chance to make img work when privilege escalation is explicitly not allowed ?

maigl commented 1 year ago

.. I found a solution that works for me: I was able to grant the required permissions by setting the needed capabilities directly on newuidmap and newgidmap

setcap CAP_SETUID+eip /usr/bin/newuidmap 
setcap CAP_SETGID+eip /usr/bin/newgidmap

within my Dockerfile.

Beware: You also need to add the capabilities to all 'calling' binaries in the call stack, e.g.

# we need to set SETUID and SETGID capabilities
# and make sure that it's inherited in all calling binaries.
RUN  setcap CAP_SETUID+eip /usr/bin/newuidmap \
  &&  setcap CAP_SETGID+eip /usr/bin/newgidmap \
  &&  setcap CAP_SETGID,CAP_SETUID+eip /usr/bin/img \
  &&  setcap CAP_SETGID,CAP_SETUID+eip /usr/bin/bash \
...

.. and after you set the capabilities you cannot copy or move the files anymore, otherwise the capabilities will get lost.