We currently run all, or at least most, of the TSS tests serially. This is done to avoid
having different tests write conflicting state to the same DBs, and to ensure each test
is using a "clean" environment.
After some of the refactoring work in #1026 where a few tests were split out, the run
time of the testing CI step went from ~45 mins to ~60 mins. This is way too long to give
developers feedback about their PR, and it also consumes more minutes on our runners (and
thus incurs extra costs).
I'm proposing that we look into running these tests in parallel by setting up an
independent network for each test. To do this we'd need to ensure that non-conflicting
ports are used for each test, as well as different database locations (for both the
Substrate nodes and TSS servers).
While we might be able to do this with the test infrastrcuture we already have, I think
it's worthwhile to look into using a combination of zombienet and cargo-nextest.
HCastano
commented
2 months ago
We currently run all, or at least most, of the TSS tests serially. This is done to avoid having different tests write conflicting state to the same DBs, and to ensure each test is using a "clean" environment.
After some of the refactoring work in #1026 where a few tests were split out, the run time of the testing CI step went from ~45 mins to ~60 mins. This is way too long to give developers feedback about their PR, and it also consumes more minutes on our runners (and thus incurs extra costs).
I'm proposing that we look into running these tests in parallel by setting up an independent network for each test. To do this we'd need to ensure that non-conflicting ports are used for each test, as well as different database locations (for both the Substrate nodes and TSS servers).
While we might be able to do this with the test infrastrcuture we already have, I think it's worthwhile to look into using a combination of zombienet and cargo-nextest.