Need for `Assert` based hooks

[AmitKumarDas]

Motivation

In the present times, Kubernetes developers need to write lot of code (w.r.t Golang) to test their controllers. It ranges from use of Ginkgo, Gomega, fake objects, standard testing package & the list continues. This issue acts as a placeholder to put thoughts to simplify this testing effort and make it more agile (when thought in terms of time taken to write this code, maintenance, rework, integrating with DAG based pipelines, integrating with CI/CD tooling, etc).

High Level Thoughts

One of the thoughts to simplify writing test code is to have Metac support Assert based hooks on the lines of Sync & Finalize hooks. Metac is already a kubernetes based level triggered reconcile system which makes it similar to Directed Acyclic Graph (DAG) based solution. In addition, if assert based hooks can enable writing assertion logic in query based language(s) that understand structured document models such as JSON, this entire process of writing test logic becomes simpler. A developer / tester needs to only assert the JSON document to test various controller scenarios.

Using Metac, a developer need to only write test logic in a declarative style. Whereas this developer gets much more in return, which are:

• Kubernetes itself becomes the CI/CD platform,
• Declarative test specs are executed as DAG
• No need to write assertion in high level languages like Golang, Python, JavaScript, etc that require specific skill set.

How to write test logic as declarative specifications?

Rego is something that I am contemplating with. However, there can be more that I am not aware of.

References

• https://github.com/instrumenta/conftest
• https://github.com/eicnix/helm-opa
• https://github.com/instrumenta/kubeval

[AmitKumarDas]

These are some of the current ways of testing kubernetes controllers:

• https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md
• https://kubernetes.io/blog/2019/03/22/kubernetes-end-to-end-testing-for-everyone/

[grzesuav]

not sure it it won't duplicate functions of https://www.openpolicyagent.org/docs/latest/kubernetes-introduction/ -- OPA Gatekeeper, which is Wehbook Validator which uses rego rules to check manifests before applying them to cluster

[grzesuav]

or you mean to write to tests for operators to be executed via metac on the cluster ?

[AmitKumarDas]

@grzesuav the second option. This is all about using metac as a conformance controller. We might use any language. However rego or jsonnet should also do this job. It should be about asserting the things in the cluster. You can think of this as a kind of continuous monitoring / health check as well.

[grzesuav]

What should happen when Assert isn't met ?

[AmitKumarDas]

Assert results will most probably be set as status & conditions against the watched resource. So users need to define their custom resource which has status & conditions & so on. This custom resource can have specs that indicates what needs to be asserted etc. The design of custom resource belongs to the end user.

GenericController will be set with above custom resource as the Watch & other resources (native or custom) as attachments. Hook logic in turn will do the checks e.g. nil checks, status checks & can have more complicated checks if required. The result of each check can be set as a condition in Watch resource. If all checks pass, then status.phase of Watch resource can be set as Healthy.

Above does not manipulate attachment resources at all. Attachment resources are read only.