Closed tspearconquest closed 1 year ago
We have similar requirements. Also using Gatekeeper, but with different constraints.
Do you know if this has been addressed or if someone is working on it?
This is something we're planning on looking into now. We would like to drop all capabilities, but it's not 100% clear to me at the moment which ones to add back. Any pointers here could save us a lot of time. Thanks.
I would start by looking at the capabilities list here: https://man7.org/linux/man-pages/man7/capabilities.7.html
Check if any of those match anything in the code of the "wrapper" that runs in the entrypoint of the container once the webhook does it's mutation.
If none match, dropping all will probably work.
For the init container, I believe dropping all should work without any issue, because it's just using the shell to copy the file into the regular container(s)
Is this something you would like to look into @Olsenius? You have access to more test clusters than me at the moment.
We are using gatekeeper on our AKS clusters and are also interested in an option to configure the securityContext for the initContainer. Related to this question: https://github.com/SparebankenVest/azure-key-vault-to-kubernetes/issues/498
@HammerNL89 I'm going to investigate soon using if we can accomplish this using Gatekeeper Mutations while we wait for the feature from the project.
We've added a pull request for this:
https://github.com/SparebankenVest/azure-key-vault-to-kubernetes/pull/548
I am good to close out this issue, and #498 can also be closed, as the feature was released in webhook-1.5.0.
For those who cannot immediately update to webhook version 1.5.0, I offer the following solution if you are using Gatekeeper 3.10.0 in your cluster. Simply apply the below manifests for Gatekeeper to override the injected container's security context (and image pull policy) via Gatekeeper's Assign
mutation resource.
Simply apply the below 2 manifests to your cluster, and restart your workloads to get the updated injection.
As a side note: You may want to also consider changing your Gatekeeper mutating webhook's reinvocationPolicy
from Never
to IfNeeded
to ensure that the mutating webhook will re-review the pod and mutate it after akv2k8s has, itself, mutated it.
apiVersion: mutations.gatekeeper.sh/v1
kind: Assign
metadata:
name: assign-akv-injected-container-image-pull-policy
spec:
applyTo:
- groups:
- ""
kinds:
- Pod
versions:
- v1
location: 'spec.initContainers[name: copy-azurekeyvault-env].imagePullPolicy'
match:
excludedNamespaces:
- kube-node-lease
- kube-public
- kube-system
kinds:
- apiGroups:
- ""
kinds:
- Pod
namespaceSelector:
matchExpressions:
- key: azure-key-vault-env-injection
operator: Exists
scope: Namespaced
parameters:
assign:
value:
Always
pathTests:
- subPath: "spec.initContainers[name: copy-azurekeyvault-env]"
condition: MustExist
---
apiVersion: mutations.gatekeeper.sh/v1
kind: Assign
metadata:
name: assign-akv-injected-container-security-context
spec:
applyTo:
- groups:
- ""
kinds:
- Pod
versions:
- v1
location: 'spec.initContainers[name: copy-azurekeyvault-env].securityContext'
match:
excludedNamespaces:
- kube-node-lease
- kube-public
- kube-system
kinds:
- apiGroups:
- ""
kinds:
- Pod
namespaceSelector:
matchExpressions:
- key: azure-key-vault-env-injection
operator: Exists
scope: Namespaced
parameters:
assign:
value:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
privileged: false
readOnlyRootFilesystem: true
runAsGroup: 10000
runAsNonRoot: true
runAsUser: 10000
seccompProfile:
type: RuntimeDefault
pathTests:
- subPath: "spec.initContainers[name: copy-azurekeyvault-env]"
condition: MustExist
- subPath: "spec.initContainers[name: copy-azurekeyvault-env].securityContext"
condition: MustNotExist
Is your feature request related to a problem? Please describe.
Pod.spec.containers.securityContext.capabilities.drop
and in their initContainers viaPod.spec.initContainers.securityContext.capabilities.drop
in the Pod or Deployment manifest.kubectl explain Pod.spec.containers.securityContext.capabilities.drop
or capabilities in the kubernetes documentationDescribe the solution you'd like
Describe alternatives you've considered
Additional context
kind: CustomResourceDefinition
which allows you to create customkind: ConstraintTemplate
.kind: Constraint
) applied on each namespace in order to define the constraints on that namespace. I will share the constraint manifest further down.We use the below Constraint Template already for any namespaces where we have exclusively pods that don't require AKV2K8S, and for namespaces which do have pods which require AKV2K8S, we have to configure our CI pipelines to not deploy this Constraint Template as well as the Constraint. Because these are not deployed to those namespaces, it reduces the security in those namespaces because containers are not forced to drop capabilities they don't need:
kubectl get pod/pod-name -n akv2k8s-pods -o yaml
on one of the pods in the akv2k8s-pods namespace, but we can only modify certain things like the image URL and tag via the environment variables on the env injector pod:I should also add that we do not necessarily have to drop
"ALL"
capabilities, as we can configure the constraint so that it will only force the dropping of the capabilities that are not required, rather than"ALL"
Additionally, I would like to point out that this is not an issue for the webhook pod itself, as we are able to modify the manifest for that pod, it is only the manifest for the environment injector which gets generated from the Go code here in this project which is unable to be modified.