Enhance e2e testing for kepler-operator

husky-parul commented 1 year ago

Currently the e2e test is very limited. It only tests on kind and only test if the kepler instances are running. The e2e tests need to be updated to cover

[x] ~~cover other env like microshift~~
[ ] openshift
[x] to support ~microshift and~ openshift we might need to investigate installation through catalog
[ ] #165

Update In the discussion on streamlining kepler-operator deployment across multiple platform like OCP, kubernetes (kind) and ~microshift~ we came to the following conclusions:

Operator installation should be done through Catalog.
In OCP environment, the user can use the community catalog to install the operator
In scenario where community catalog is not available (~microshift~air-gapped etc) users will deploy kepler by building local file catalog or adding the CatalogSource to their cluster.

This will ensure consistent deployment across all platforms and ensure that all deployments are tested via the e2e tests in the CI.

Following issues will be also addressed:

[x] #104

This issue is relies on operator-sdk in local dev cluster to run olm suggested in this issue https://github.com/sustainable-computing-io/local-dev-cluster/issues/39

@sustainable-computing-io/keperl-ci-maintainer @sustainable-computing-io/maintainer @sthaha @vprashar2929

husky-parul commented 1 year ago

@vprashar2929 @vimalk78 We can use this issue for e2e testing of the operator

husky-parul commented 1 year ago

@vprashar2929 @sthaha What fw can be use in the upstream. Initially we started with kuttl but since it is limited to kind we didn't make much progress. If these tests are yaml based (both in the upstream and downstream) we can re use the artifacts in that case. Wdyt?

sthaha commented 1 year ago

What we did in ObO was to just go test itself to write all e2e tests. Writing tests are fairly straight forward and the test we write isn't tied to any particular flavour of k8s.

see: https://github.com/rhobs/observability-operator/blob/9b663b73963cf94a91b784680eb5b0cf4aa7c41a/test/e2e/monitoring_stack_controller_test.go#L214

husky-parul commented 1 year ago

@vprashar2929 Can you please advise on how we can do tests for OpenShift using their CI? (If we can get openshift-ci integrated with the sustainble-computinng org)

sthaha commented 1 year ago

cover other env like microshift

We are removing microshift since we intent to have multiple nodes available in our CI which is only supported by kind.

sustainable-computing-io / kepler-operator

Enhance e2e testing for kepler-operator #87