search

kurtosis-tech / kurtosis

A platform for packaging and launching ephemeral backend stacks with a focus on approachability for the average developer.
https://docs.kurtosistech.com/
Apache License 2.0
351 stars 47 forks source link

GitHub Actions Workflow #2284

Open igor-ritual opened 6 months ago

igor-ritual commented 6 months ago

Background & motivation

I am looking to run tests for a kurtosis package repository using GitHub Actions. A couple of issues arise:

Desired behaviour

The kurtosis cli should support run and block while running a package and terminate with an error number given the package script.

How important is this to you?

Critical; Kurtosis is unusable for me without it.

What area of the product does this pertain to?

CLI: the Command Line Interface

galenmarchetti commented 6 months ago

hey @igor-ritual thanks for filing this! this is a great use case for kurtosis and I'd love to make this work for you. The way most folks are doing this right now is via a combination of the CLI and using our Golang or TS SDKs for interacting with the kurtosis engine to handle testing assertions and teardowns. I'll give some thoughts on each bullet point you mentioned:

With regards to stopping and tearing down an enclave from within the package script - we've designed the enclave stop and enclave rm functionality to happen outside of the packaging system, meaning they're called from either the CLI or directly from the Engine & APIC protobuf apis (https://docs.kurtosis.com/engine-apic-reference), for which we have a TS client library and a Golang client library.

So typically in a workflow like a github actions workflow, users write either a bash script that uses the CLI to stop and tear down enclaves, or they call into the TS or Golang client libraries to run tests in those languages, and tear down the enclaves at that layer as well.

Do those options work for handling the first issue of "how to stop an tear down the enclave from within the package script"? or am I missing something about your use case that requires stopping and tearing down the enclave within the package script specifically - is there a reason it has to happen within the script?

With regards to the second bulletpoint "How to signal any errors back to the workflow if any tests fail?" - how are you running assertions from your tests? If you use the TS or Golang client libraries then your test assertions would be in the language specific testing libraries and you would be able to use those SDKs to trigger shutdown of the enclaves.

I might be totally off the mark in understanding your workflow and what you're trying to do here, so please let me know if the above context was helpful in achieving your workflows, or if it's not, could you elaborate on how the "run and block in the CLI, along with error codes" will unblock your testing use case so I can understand better?