Closed MarkusFlorian79 closed 7 months ago
Are you having issues clearing that default? Although Helm's default value system is cumbersome to work with, an explicit null
does appear to clear this:
$ helm template ana kong/kong --set containerSecurityContext.runAsUser=null | grep -iA9 securitycontext | head -11
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
--
As #909 states, this doesn't have anything to do the reason mentioned in #1003
Versions of Kong prior to 2.0 and Kong Enterprise prior to 1.3 use Docker images that required setting a UID via Kubernetes in some environments
This is for compliance with Kubernetes 1.23+ default Restricted security policy (under "Running as Non-root user") to not run as root. Although docs state that nil should be allowed, the full story is more complex. The code check for this has failure cases for nil UID.
IIRC in practical testing I could not deploy into a restricted-enforced environment without adding the explicit run as user. I'm not familiar with how the username input to that verification function is derived, I expect it's looking at the container metadata and seeing the kong
user we create, whose UID is determined by the distro package, not the container image build.
We are using the chart as subchart. Setting the runAsUser=null results in the default value 1000 being used.
Testing confirms we do indeed hit that restriction if we remove this. To demonstrate, I created a restricted namespace and installed several configurations into it and default
(which has no policy configured):
48853 kubectl create -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/security/podsecurity-restricted.yaml
48855 helm install unrestricted-null kong/kong --set containerSecurityContext.runAsUser=null
48856 helm install restricted-default -n my-restricted-namespace kong/kong
48857 helm install restricted-null -n my-restricted-namespace kong/kong --set containerSecurityContext.runAsUser=null
48859 helm install restricted-null-asroot -n my-restricted-namespace kong/kong --set containerSecurityContext.runAsUser=null --set containerSecurityContext.runAsNonRoot=false
Removing it actually breaks even the unrestricted install. The check actually confirms whether the container complies with its runAsNonRoot
setting regardless of whether the namespace policy requires it:
11:13:05-0800 esenin $ kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
httpbin 1/1 1 1 36m
unrestricted-null-kong 0/1 1 0 20m
11:13:12-0800 esenin $ kubectl describe po unrestricted-null-kong-7c464b8c6f-57kzj| grep -i root
ConfigMapName: kube-root-ca.crt
Warning Failed 19m (x12 over 21m) kubelet Error: container has runAsNonRoot and image has non-numeric user (kong), cannot verify user is non-root (pod: "unrestricted-null-kong-7c464b8c6f-57kzj_default(dabf4849-283d-4956-8bb1-60f9ad00d09a)", container: clear-stale-pid)
In the restricted namespace you'll get the same failure if you install with runAsNonRoot=true
and no UID. You'll additionally get a failure on the ReplicasSet if you toggle runAsNonRoot
off because it violates policy. Though not shown, installing with --set containerSecurityContext.runAsUser=null --set containerSecurityContext.runAsNonRoot=false
in default does work:
11:13:28-0800 esenin $ kubectl get deploy -n my-restricted-namespace
NAME READY UP-TO-DATE AVAILABLE AGE
restricted-default-kong 1/1 1 1 20m
restricted-null-asroot-kong 0/1 0 0 17m
restricted-null-kong 0/1 1 0 20m
11:13:44-0800 esenin $ kubectl describe po -n my-restricted-namespace restricted-null-kong-65ddbd6947-tpxdq| grep -i root
ConfigMapName: kube-root-ca.crt
Warning Failed 18m (x12 over 20m) kubelet Error: container has runAsNonRoot and image has non-numeric user (kong), cannot verify user is non-root (pod: "restricted-null-kong-65ddbd6947-tpxdq_my-restricted-namespace(3dc95a4c-cd06-45f0-8269-fa25bdd2507b)", container: clear-stale-pid)
11:14:41-0800 esenin $ kubectl describe replicasets.apps -n my-restricted-namespace restricted-null-asroot-kong | grep -i forbidden | head -1
Warning FailedCreate 18m replicaset-controller Error creating: pods "restricted-null-asroot-kong-7cf76d44d7-88r8z" is forbidden: violates PodSecurity "restricted:latest": runAsNonRoot != true (containers "clear-stale-pid", "ingress-controller", "proxy" must not set securityContext.runAsNonRoot=false)
So AFAICT nulling the setting is the only option if we don't want to remove restricted compliance and further indicate that the container may run as root.
The implementation of the check suggests that there may be a way to create a container with no user that then passes the check, but I suspect that's not actually possible and is just an entirely hypothetical situation the code can handle. I've asked SIG Auth if they know otherwise, but in any case I think it's likely we'd clear the user from the container image build.
We are using the chart as subchart. Setting the runAsUser=null results in the default value 1000 being used.
Fascinating. The subchart implementation never fails to amaze: https://github.com/helm/helm/issues/12637
It looks like the Pod enforcement can indeed permit runAsNonRoot=true
without a UID if you use a numeric ID at build time, so patching one of the standard Dockerfiles and building gets you something that's compatible:
$ git diff
diff --git a/Dockerfile.deb b/Dockerfile.deb
index b367869..34f65a6 100644
--- a/Dockerfile.deb
+++ b/Dockerfile.deb
@@ -39,7 +39,7 @@ RUN set -ex; \
COPY docker-entrypoint.sh /docker-entrypoint.sh
-USER kong
+USER 1000
ENTRYPOINT ["/docker-entrypoint.sh"]
Having built that, in the unrestricted namespace:
$ helm install uuu kong/kong --set containerSecurityContext.runAsUser=null --set image.repository=tmp --set image.tag=tmp --set image.effectiveSemver=3.5.0
$ kubectl get po
NAME READY STATUS RESTARTS AGE
boo-kong-d7d68dcbb-hbvv5 2/2 Running 0 30m
httpbin-66c5b5bc6b-xnjxp 1/1 Running 0 74m
unrestricted-null-kong-5bf5b64798-mzpr2 0/2 Init:CreateContainerConfigError 0 11m
uuu-kong-57ffbc5bfb-ssx2v 2/2 Running
The controller already uses a numeric USER argument, so it doesn't need a custom image.
It looks like you should also be able to instruct OpenShift that some given range of UIDs is acceptable for a namespace (or project). Is that an option? It's probably simpler than maintaining a custom image, even if it's a simple one.
https://www.ibm.com/docs/en/cloud-paks/cp-management/2.2.x?topic=collection-adding-user-id-group-id-in-dockerfile https://docs.openshift.com/dedicated/authentication/managing-security-context-constraints.html#security-context-constraints-pre-allocated-values_configuring-internal-oauth https://developer.ibm.com/learningpaths/secure-context-constraints-openshift/deployment-specify-permissions
Some of that content suggests you can maybe set this permitted range at the Deployment level, but I'm not sure. If that's actually allowed, we could probably add those annotations to the chart resources.
Thx for your efforts. For the time being we will set the runAsUser explicitly, even if it could change after recreation of the project.
Openshift does dynamically attribute defined user ranges. Therefore the user 1000 is in general not a valid user. In version 1.6 it was removed already (### Removal of default security context UID setting) , but was reintroduced with #909. The securityContext can be defined without the runAsUser and everything works fine.