maelstrom-software / maelstrom

Maelstrom is a fast Rust, Go, and Python test runner that runs every test in its own container. Tests are either run locally or distributed to a clustered job runner.
https://maelstrom-software.com/
Apache License 2.0
600 stars 11 forks source link

Test runners should show failed tests before summary. #391

Closed nfachan closed 2 months ago

nfachan commented 3 months ago

Use Cases

When I'm running a large batch of tests, it's possible that it takes a while, but also that some of the test fail early on. I want to know that some tests have failed without waiting until the end of the test run.

Acceptance Criteria

Implementation Considerations

Definition of Done

bobbobbio commented 2 months ago

Optionally, if there are only one or two failures, and if there is room, the first bit of stderr should be included so that I can start fixing it while the tests continue to run.

I think this needs some more thought. The relevant part of the output isn't clear and on my terminal there really isn't enough room to show anything all that useful. We could show it if there is only one failure, but I'm wondering if instead we want to use keyboard controls to let you choose a failure whose output to look at or something along those lines. Something like a collapsible list item