Ability to run devpods as non-root or even in restricted pod security standard environments

[chgl]

Currently, at least the init container seems to require running as root (UID 0, see https://github.com/loft-sh/devpod-provider-kubernetes/blob/05c6b063f3548b40218b3ade6d0618d4e7337b9e/pkg/kubernetes/init_container.go#L45) with no way to configure it.

It would be great if we could deploy devpod to clusters that require running as non-root. Even better, to clusters that use the restricted pod security standard: https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted.

Is this something that is technically possible?

[solomonakinyemi]

This is a feature I am also trying to navigate around too.

• runAsNonRoot
• runAsUser
• allowPrivilegeEscalation: false
• capabilities (drop: ALL)

[chgl]

I played around just setting runAsNonRoot: true and specifying a non-0 UID, but I quickly ran into filesystem permission issues, specifically a failure to create the /var/run/devpod folder (see https://github.com/loft-sh/devpod/blob/fd6b4d35664353087c704ed9ba895dda7ff21696/pkg/devcontainer/setup/setup.go#L22 for quite a few chown and similar commands that won't work as non-root). From a very cursory look, I think it could be possible to run as non-root with an existing user if any global changes are moved to the home dir instead (~/.profile vs /etc/profile; /run/user/1000/devpod vs /var/run/devpod). Not sure how portable that is though.

[pascalbreuninger]

@chgl One of the problems here is that the init container is based on the workspace container, so we can't know in advance whether root is needed or not. For some cases it would work for sure, maybe that's something we could solve for in a provider option or a devcontainer customization

[solomonakinyemi]

@chgl does the init container have to run as root? I was trying to use this: https://github.com/loft-sh/devpod/pull/547, but no luck.

[chgl]

@pascalbreuninger You're right, it will definitely need to be opt-in. I think the base requirement can be a base image with a non-root user with a writeable home directory - the location for any persistence/markers that devpod requires.

@solomonakinyemi thanks for sharing! I wasn't aware of that. But it looks like the init-container is currently hard-coded to run as root, see: https://github.com/loft-sh/devpod-provider-kubernetes/blob/05c6b063f3548b40218b3ade6d0618d4e7337b9e/pkg/kubernetes/init_container.go#L42-L45. The same is true for the devpod/workspace (?) container at https://github.com/loft-sh/devpod-provider-kubernetes/blob/05c6b063f3548b40218b3ade6d0618d4e7337b9e/pkg/kubernetes/run.go#L238-L240. So it currently does run as root, whether it has to, I am not sure. Probably not if both the init and the workspace container run as the same UID.

[sanmai-NL]

@pascalbreuninger Running as an unprivileged container user is a feature of Visual Studio Code dev containers. What does this imply for implementation within DevPod?

[pascalbreuninger]

@sanmai-NL wow, I totally missed this question - sorry. We can add a non-root mode for sure, it's just that it has to be opt-in because you need to be aware of the consequences this might have:

1. We can't build images in-container anymore afaik (need to test this), meaning you'd have to build it first, push to a registry and then DevPod can pick it up
2. Some features expect to be root and privilege escalation, like docker-in-docker. You'd have to be aware of that ahead of time to adjust your workflow

[sanmai-NL]

1. You can build images using rootless Podman. There's a Podman container image that offers this out the box, and I use it in CI/CD.
2. I'm a bit unsure how relevant this feature and the general problem is. People have to be aware of what they're doing, but that's in general. From your wording you seem to think non-root container users are exotic or unexpected, where they are the default nowadays.

[pascalbreuninger]

@sanmai-NL Makes sense 👍 We've added a new STRICT_SECURITY provider option with #36 and going to cut an alpha release for this. You will need to add the alpha provider with devpod provider add kubernetes@v0.1.8-alpha.0

Setting the option to true (default: false) will remove the security context from the DevPod container and merge any additional security options from the POD_MANIFEST_TEMPLATE if it is specified.

Let us know how it goes for you!