Within a test set, we need to be able to interleave tests. This will require a new 'interleave' option in series (at both the top and per test_set level).
The most straightforward way to do this is add an option to the 'load()' and 'load_iter()' Resolver object methods, that force the resolver to juggle between each requested test. The end result should be that requests are cycled through, yielding one test per request.
So given requests for tests A, B, C, where A has 4 permutations, B has 4 permutations, and C has 2 sub-tests, We would see:
A.1
B.1
C.a
A.2
B.2
C.b
A.3
B.3
A.4
B.4
Part 2 of this is to add a separate series option to randomize node order for per-node permutations. This might actually be achievable already with chunk.size=1, chunk.selection=random.
Within a test set, we need to be able to interleave tests. This will require a new 'interleave' option in series (at both the top and per test_set level). The most straightforward way to do this is add an option to the 'load()' and 'load_iter()' Resolver object methods, that force the resolver to juggle between each requested test. The end result should be that requests are cycled through, yielding one test per request.
So given requests for tests A, B, C, where A has 4 permutations, B has 4 permutations, and C has 2 sub-tests, We would see:
Part 2 of this is to add a separate series option to randomize node order for per-node permutations. This might actually be achievable already with
chunk.size=1, chunk.selection=random.