search

powdr-labs / powdr

A modular stack for zkVMs, with a focus on productivity, security and performance.
Apache License 2.0
356 stars 72 forks source link

Improve our use of nextest #1277

Open chriseth opened 3 months ago

chriseth commented 3 months ago

Nextest has some features that might be helpful but which we do not yet use:

1) marking tests as needing multiple threads

Currently, we run all of the tests with a single thread in our nighly run because some of them need a lot of memory. Instead, we could use nextest and just mark the heavy tests as needing 8 or 16 cpus which should cause them being run in isolation, but the non-heavy tests are still run in parallel.

2) generating junit output

Nextest can generate junit output which we could process to get a nice summary of test success and failures (via an appropriate github action script). This way, we could get a list of failing tests directly on the PR page and don't need to go to the action report.

3) tune the slow-ness report

Many of our tests are marked as slow. The time threshold for tests being marked as slow is configurable, though.

lvella commented 3 months ago

I don't like number 2. When the main PR interface is used by bots and commands, it tends to become too cluttered, burying the conversations in bot spam.

chriseth commented 3 months ago

True. Maybe there is a way to have at most one comment that contains the test results and is updated for a new run.