Monokle policies as CRDs

[f1ames]

This PR fixes https://github.com/kubeshop/monokle-saas/issues/1839, fixes https://github.com/kubeshop/monokle-saas/issues/1845.

Implemented MonoklePolicy and MonoklePolicyBinding according to https://github.com/kubeshop/monokle-saas/issues/1839#issuecomment-1738937312 and consistent (to some extend) with how VAP works.

Changes

• Introduced CRDs for MonoklePolicy and MonoklePolicyBinding.
  • VAP supports 3 action types. For the first version we decided to go with Warn only (see screenshot below). I already put it in the CRDs definition (as enum array so it can be extended and is the same format VAP has).
• Introduced PolicyManager class which watches instances of MonoklePolicy and MonoklePolicyBinding and is responsible for storing, updating and matching (policy-binding-resource) them (I think *Manager class name is a bit of naming-antipattern so I'm open for suggestions).
  • This is based on K8s Informer (high level watcher) mechanism.
• Introduced ValidatorManager class which reacts to policy changes and creates/updates/deletes validator instances. It also provides matching validators for given resources when they need to be validated.
• Reworked the structure of the project. Also, by default all namespaced resources related to admission controller deployment will be deployed to monokle-admission-controller namespace.
• Added mechanism blocking plugins which will not work at the current state (resource-links). See https://github.com/kubeshop/monokle-saas/issues/1845#issuecomment-1744412694 and https://github.com/kubeshop/monokle-admission-controller/pull/1/commits/586aeed14a9366e68d087de3a5be2586f6bd6aaa.

Sample output (we can adjust wording there):

Fixes

• Mostly major changes as described above.

To consider

• Namespace matching - see this comment.
• Revisit logs - I used pino for logging and try to make it reasonable with different levels (so it's not to much with info and higher level while debug/trace will log a lot of data on each update, like resources content, polices content, etc). Still it might require some improvements to better help with debugging.
• Deployment part (via Helm) is not there yet since we have separate task for it - https://github.com/kubeshop/monokle-saas/issues/1842.

I'm open for discussion/suggestions for any of the above.

Testing

See README.md with step-by-step instructions.

Checklist

• [x] tested locally
• [x] added new dependencies
• [x] updated the docs
• [ ] added a test

[olensmar]

I really think we should reconsider our approach on namespaced admission controllers. Or maybe i did not understand the code and I might be wrong, but the approach where the controller can have an one "active policy" at a time is not looking reliable to me.

A better aproach, on a quick thought, would be creating a Map, 1-1 between a namespace and a policy (at first, later we can map resources to policies better should we want that and if it is possible). Now the Controller would have knowledge of all the policies and would be able to validate namespaced manifests.

totally agree - this would be configured in cloud, i.e. which policies apply to which resources - the selection could be based on

• namespace
• labels/annotations
• resource kinds
• ?

[f1ames]

Ok, discussed some technicalities with @murarustefaan to make sure I understand how admission controller should behave exactly, to summarize:

• Admission controller should be global (not namespaced) and should react on resources creation/updates across namespaces (this will be configurable later, which namespaces, etc).
  • Still, webhook deployment/pod will belong to a namespace (should be configurable during deployment).
• Based on resource data, it will use policy from the same namespace to validate it.
  • Should we fallback to any default if there is no policy in a given namespace?
• When integrating with Monokle Cloud, it should allow configurabilty, to define which policies should be applied to which resources (see https://github.com/kubeshop/monokle-admission-controller/pull/1#issuecomment-1737073914).
  • I already added some initial thoughts on configurability here - https://github.com/kubeshop/monokle-saas/issues/1839#issuecomment-1735503488. Suggestions welcome 👍