Study and document policy priority rules and model

[viccuad]

Description

Document the several models on using non-mutating and mutating policies altogether.

Currently, the defined order of execution of Kubewarden policies is just the order that the webhooks get registered against the server (first mutating webhooks in the mutating admission phase, then validating wehooks in the validating admission phase). Since our policies are more atomic than PSPs for example, we don't have the problems of PSPs where 2 PSPs make big mutating changes to several fields at once.

Kubewarden shouldn't completely disallow approaches different than the current by being too opinionated. The community should be the one coming up with standards for the policy engines.

Example

With a default Kubewarden installation, deploy the default policies in protect mode:

$ helm upgrade --install --wait --namespace kubewarden --create-namespace --set recommendedPolicies.enabled=True --set recommendedPolicies.defaultPolicyMode=protect kubewarden-defaults kubewarden/kubewarden-defaults

Now, try to instantiate a privileged pod. The policy no-privilege-escalation kicks in, mutates the pod, setting allowPrivilegeEscalation false, which is incompatible with the preexisting privileged: true. This is ok, but showcases the order when applying policy rules:

$ kubectl run pod-privileged --image=k8s.gcr.io/pause --privileged
The Pod "pod-privileged" is invalid: spec.containers[0].securityContext: Invalid value: core.SecurityContext{Capabilities:(*core.Capabilities)(0xc00a1d65d0), Privileged:(*bool)(0xc010c2e6ba), SELinuxOptions:(*core.SELinuxOptions)(nil), WindowsOptions:(*core.WindowsSecurityContextOptions)(nil), RunAsUser:(*int64)(nil), RunAsGroup:(*int64)(nil), RunAsNonRoot:(*bool)(0xc010c2e6bb), ReadOnlyRootFilesystem:(*bool)(nil), AllowPrivilegeEscalation:(*bool)(0xc010c2e6b6), ProcMount:(*core.ProcMountType)(nil), SeccompProfile:(*core.SeccompProfile)(nil)}: cannot set `allowPrivilegeEscalation` to false and `privileged` to true

PSPs

For PSPs, it works such as:

1. find a non-mutating policy that matches the request, if it allows the request to pass, use it.
2. If there's no non-mutating policy that allows the request, go to the mutating ones. Try to apply them in alphabetical order.

This informs a mental model such as:

1. Either use application-specific PSPs, that ship with the application (helm chart), or target only the application ns, and are written to match the specific application (open ports, volumes, whatever), and require no mutation.
2. Or apply catch-all PSPs that mutate, and provide a basic level of security if things are not checked by the application (what we do with our policies-substituting policies)

PodSecurity

PodSecurity don't perform mutation, and simply act as gating policies for the namespaces.

By deciding to disallow mutating policies at the policy-server level (or just deploy all policies with mutating: false), non-mutating policies would be the first and only to run, and would emulate the PodSecurity behaviour.

[flavio]

I'm confused, what is the end goal of this card? Kubewarden builds on top of Kubernetes' Dynamic Admission Controllers, which all suffer from the ordering and namespace issues.

Can we define a clear scope for this documentation efforts? I fear we would end up writing generic Kubernetes docs.

[viccuad]

Maybe:

• Reword the first paragraphs of https://docs.kubewarden.io/tasksDir/psp-migration , to explain how PSPs are normally used (either non-mutating application specific, or mutating, catch-all ones). And how Kubewarden can be configured for both cases.
• Add a new short docs page on PodSecurity, and how to use Kubewarden to follow that model, or together with PodSecurity (use non-mutating AdmissionPolicies). Remind people that Kubewarden allows to use ClusterAdmissionPolicies and mutating ones on top, which would apply first.

[raulcabello]

Reword the first paragraphs of https://docs.kubewarden.io/tasksDir/psp-migration , to explain how PSPs are normally used (either non-mutating application specific, or mutating, catch-all ones). And how Kubewarden can be configured for both cases.

I wouldn't explain here how PSPs are normally used, which is confusing (one of the reasons that they got deprecated). PSPs are deprecated and should not be used anymore. This page is just about how to migrate from PSPs to Kubewarden. It is already quite a long page. Therefore, I wouldn't add anything that doesn't help with running the migration script.

Add a new short docs page on PodSecurity, and how to use Kubewarden to follow that model, or together with PodSecurity (use non-mutating AdmissionPolicies). Remind people that Kubewarden allows to use ClusterAdmissionPolicies and mutating ones on top, which would apply first.

A new short docs page on PodSecurity and how it can be used with Kubewarden make sense to me.

[flavio]

I agree with @raulcabello, we should instead write something about how PodSecurity and Kubewarden can complement each other (especially because PodSecurity is not a 1:1 replacement of PSP)