maelstrom-software / maelstrom

Maelstrom is a fast Rust, Go, and Python test runner that runs every test in its own container. Tests are either run locally or distributed to a clustered job runner.
https://maelstrom-software.com/
Apache License 2.0
600 stars 11 forks source link

Provide some way to cache test results for deterministic tests #400

Open bobbobbio opened 3 months ago

bobbobbio commented 3 months ago

Use Cases I want to save time by not running deterministic tests which we know pass from prior running of it. I want this to just happen for me automatically. This database of cached test results I want to somehow reuse across automation runs to save time.

This isn't super well-defined yet, this is mostly a placeholder.

There are some serious tricky things to consider when trying to figure out if a test is deterministic or not.

This probably depends on #381

Definition of Done