should enforcement trigger on non-pod resources?

[mrparkers]

right now, we only concern ourselves with pods when evaluating resources within rode. however, several kubernetes resources will manage pods for you; rarely do users ever create pods directly.

this leads to all sorts of less than ideal situations. for example, a user might create a deployment with three replicas, which causes three resource evaluations to trigger against the same docker container. this technically "works", but it's obviously a performance hit, and it will clutter the UI with a bunch of meaningless evaluation data.

the other downside to this is that users who create deployments (or even CI jobs that create deployments) won't actually know whether or not the underlying pods are deployed successfully without manually checking after creating the deployment. the deployment will create successfully, which will cause a replicaset to create successfully, which will then cause x pods to fail to create.

here's a list of the k8s resources that I'm aware of that manage pods for you:

• deployments
• replicasets (typically created by deployments)
• statefulsets
• daemonsets

how do we want to handle these? we could update the enforcer to handle all pod managing resources, but this alone doesn't solve all of the potential issues. in the deployment example, we can trigger the enforcer on the deployment and evaluate the container in the deployment spec, but then we'd have to do the same thing for the replicaset and the underlying pods. so a deployment with three replicas would still result in five resource evaluations.

when admitting a k8s resource, we could check if the resource has an owner ref in order to only evaluate the policy once, but we'd have verify somehow that the topmost resource actually ran and passed policy. maybe we need to update the EvaluateResource RPC to accept metadata that the enforcer can search on in order to determine if policy needs to be re-evaluated.

[ChrisSchreiber]

I feel like this is trying to solve a larger problem with k8s CD pipelines that might be better handled another way. There are lots of reasons that a deployment resource is successfully created and the pods are not run (insufficient resources, problems mounting a volume, problems pulling the image). You can use something like prometheus alerts to get visibility to the problem but that is still not ideal for the pipeline but that is not a problem unique to Rode policy evaluation.

I also don't think it is unreasonable for the pipeline to explicitly test the policy before deploying or check that the application is actually running after deploying independently of k8s enforcer testing the policy before running a container.

In terms of performance and storage issues I think we should focus on scaling Rode and possibly caching result internally (see https://github.com/rode/enforcer-k8s/issues/16#issuecomment-872416163). We might also need to add the ability to collapse duplicate data in the UI.

Even if we handle all of the resources above (plus jobs) it still wouldn't handle custom resources the manage pods.