search

spinkube / runtime-class-manager

A Kubernetes operator to manage Runtime Classes
Apache License 2.0
36 stars 10 forks source link

ci(.github): add helm chart smoketest workflow #234

Closed vdice closed 1 month ago

vdice commented 1 month ago

Describe your changes

Adds initial helm chart test automation using a KinD cluster.

Ref https://github.com/spinkube/runtime-class-manager/issues/48

Issue ticket number and link

Checklist before requesting a review

vdice commented 1 month ago

~Hmmm, I'm wondering why the main CI workflow (where the new chart test is added, along with preexisting unit and lint jobs) isn't running for my PR 🤔~ edit: fixed, had an error in ci.yaml, now updated

vdice commented 1 month ago

Seeing a failure where the shim isn't installed or can't be located, etc. I'm guessing this isn't the test automation; I'll test the usual helm chart install off of main onto KinD to compare notes/debug...

vdice commented 1 month ago

I'm guessing this isn't the test automation

Lol well it mostly was. Needed to update the shim binary arch to match the GH runner's arch (amd64/x86_64).

Ready for review now!

voigt commented 1 month ago

Really nice to have some chart testing! Also the PR looks pretty good to me.

Nevertheless a point for discussion: while I personally also mainly use kind for developing and testing locally, the spin-operator seems to use k3d for all pipeline-based tests. Should we align ourselves here?

From a technical PoV I'm not 100% sure how much of a technical difference this is... wdyt?

vdice commented 1 month ago

Nevertheless a point for discussion: while I personally also mainly use kind for developing and testing locally, the spin-operator seems to use k3d for all pipeline-based tests. Should we align ourselves here?

Thanks @voigt, that's a great question. I should have mentioned my thoughts on that in this PR. Spin Operator indeed uses the k3d image shipped by the spinkube/containerd-shim-spin project both in its test automation and quick start due to efficiency: the spin shim is already baked in to the image and so will exist on all nodes based on the image. Since installing and configuring the shim is part of this project's duties and therefore functionality we'd like to test, I was thinking a blank-slate cluster like KinD might be a good first start.

All that being said, as laid out in https://github.com/spinkube/runtime-class-manager/issues/48, the ultimate vision is to have test coverage for all supported distros and k3d (either with shim pre-installed or stock) does seem like a good next one to add, as well as other free/easy options like minikube, etc. So let's definitely think about our preferred approach for supporting the full matrix of distros. I think we'll still maybe want one quick/easy 'smoke test' for the helm install, ideally on a lightweight distro like kind, to automatically run on PRs... and then maybe the full suite for merges to main?

For "preferred approach for supporting the full matrix of distros", do we see GH workflow yaml(s) the right spot to orchestrate? Or something higher-level/custom eg a go binary, etc? (This could be good discussion to also have back in #48)

voigt commented 1 month ago

So let's definitely think about our preferred approach for supporting the full matrix of distros. I think we'll still maybe want one quick/easy 'smoke test' for the helm install, ideally on a lightweight distro like kind, to automatically run on PRs... and then maybe the full suite for merges to main?

I fully agree.

For "preferred approach for supporting the full matrix of distros", do we see GH workflow yaml(s) the right spot to orchestrate? Or something higher-level/custom eg a go binary, etc? (This could be good discussion to also have back in https://github.com/spinkube/runtime-class-manager/issues/48)

I don't have a good answer yet. In any case, this question probably exceeds the scope of this PR; let's take it to one of the next SpinKube community meetings.