Support post deployment test integration

[srfrnk]

Add support for following use case:

[srfrnk]

Things the post deployment step might need to do:

• Wait/Trigger on a condition such as:
  • Enough time passed since deployment
  • Deployment is stable (i.e. no changes within interval?)
• Run "standard K8s" tests such as:
  • All resource status is "green"
  • No error Events on resources(?)
• Run customized tests (bring your own test pod?)
• Mark failed/passed with Event
• Set passing status along with latest commit status

[srfrnk]

• [ ] ***Manifest resources - status lastTestedCommit and Tested condition added
• [ ] ***Manifest resources - add passConditions
• [ ]

[srfrnk]

available options

• kubectl rollout status deployment myapp (and configured with these)
• life-cycle probes
• helm tests?
• octopus
• kube-monkey
• kube-hunter
• kube-burner
• kube-bench
• kubetest2
• e2e-framework
• Bash Automated Testing System
• DETIK
• KIND?

[srfrnk]

After some research

It looks like there is no standard. Many solutions are available some more automated than others. All solutions require lots or scripting to integrate with other solutions and CI/CD frameworks.

Thus a solution here must include

1) Waiting for pre-condition 1) Triggering a custom workload (with any testing solution preferred) 1) Waiting for testing workload to complete 1) Determining outcome and configuring status accordingly

pre-condition, completion-condition and outcome-condition

1) Time since ??? 1) Readiness of ??? 1) Annotation check 1) Label check 1) Status check

All of these may be possible using jq or yq given a resource definition.

Custom workload

1) Container 1) Pods 1) Jobs/Deployments 1) More complex setups - e.g. services/roles etc...

While being under permissive is not desired - over complicating the setup may also be counter productive. Allowing a Pod definition might be a good comromise.