search

kudobuilder / kuttl

KUbernetes Test TooL (kuttl)
https://kuttl.dev
Apache License 2.0
673 stars 86 forks source link

AfterSuite Feature #292

Open faiq opened 3 years ago

faiq commented 3 years ago

What would you like to be added: AfterSuite for global test teardown. I am copying this idea from the Ginkgo test framework, which explains it in depth in it's documentation

From the documentation:

The AfterSuite function is run after all the specs have run, regardless of whether any tests have failed. Since the AfterSuite typically includes code to clean up persistent state ginkgo will also run AfterSuite when you send the running test suite an interrupt signal (^C). To abort the AfterSuite send another interrupt signal.

Why is this needed:

Having this functionality will allow us to run steps regardless of whether or not the previous assertions have failed. I'd use this to be able to do things like gather information from a cluster in the case my test fails. A lot of the tests we run with KUTTL are not executed on KIND clusters and instead on real clusters, so we can't leave around a kind cluster to analyze later.

alenkacz commented 3 years ago

Oh yeah, I need this as well :D beware that some part of this was already described in this KEP https://github.com/kudobuilder/kuttl/blob/main/keps/0006-lifecycle.md just for reference (we don't need to follow that design but new KEP should at least reference it)

faiq commented 3 years ago

Right now I'm accomplishing this code https://gist.github.com/faiq/fca924095ad151eac38b0647480c0da0 - basically calling the kuttl tests from Go and adding some go code to do the clean up for me. With the way kuttl works, we'd need to delegate these out like we do commands in TestSteps, but I would imagine setting t.cleanup would be the way of doing it.

It would be good to get a good understanding of how far the KEP you mentioned is implemented because we could include the hooks for BeforeSuite AfterSuite BeforeStep AfterStep etc. This also seems related to #294 as we could accomplish waiting with the BeforeSuite as well

kensipe commented 3 years ago

love this... I will schedule for next release... but that is pretty full (based on expected dev involvement). Would love a PR if you have the time.

egegunes commented 1 year ago

Hey @kensipe, any chance to add this to 0.15.0?