search

UnkindPartition / tasty

Modern and extensible testing framework for Haskell
637 stars 108 forks source link

It would be nice to be able to limit the number of jobs used by `TestTree`s #406

Open michaelpj opened 5 months ago

michaelpj commented 5 months ago

From looking at the code and experimentation, I believe that the NumThreads option only takes effect at the top-level. So you can't, for example, limit a particular test tree to be run with only one job (in case the tests interfere with each other).

carbolymer commented 4 months ago

I think sequentialTestGroup can be used for enforcing sequential execution on more granular level. Haven't tested how it behaves if you mix it with testGroup.

michaelpj commented 4 months ago

And that led me to ExecutionMode, which seems like it's exactly what I want but sadly isn't exported. Maybe there's a reason why it's only available through sequentialTestGroup, but I'm not sure :thinking:

michaelpj commented 4 months ago

Indeed, sequentialTestGroup does not do what I want, since it only makes the tests directly in the group run sequentially, whereas tests in nested groups continue to execute in parallel. I really do just want to set the execution mode to sequential for a whole test tree!

Bodigrim commented 4 months ago

@michaelpj it is admittedly awkward, but you can pattern-match on 

PlusTestOptions mkSequential = sequentialTestGroup "foo" AllFinish []

and then

defaultMain $ PlusTestOptions mkSequential testTree

I'm curious if it works, please tell if you give it a try.

dyniec commented 1 month ago

contructors of TestTree aren't exported, so PlusTestOptions is not visible :/

Bodigrim commented 1 month ago

@dyniec constructors of TestTree are exported from https://hackage.haskell.org/package/tasty-1.5/docs/Test-Tasty-Runners.html#t:TestTree