search

cyrusimap / cassandane

Other
6 stars 11 forks source link

tests skipped using version attributes report as "ok" during parallel runs #7

Closed elliefm closed 7 years ago

elliefm commented 7 years ago

cf:

./testrunner.pl -f prettier Skip

vs:

./testrunner.pl -f prettier -j 4 Skip

The tests are not actually run (as evidence by the Skip tests, which fail if they're run when not supposed to), they just report OK instead of reporting nothing.

elliefm commented 7 years ago

This seems to come from them being assigned into a worker pool in the parallel case. The code path for running the test skips appropriately (no idea where though), but because it's in the pool we still "finish" the workitem.

This line is responsible for producing the "ok" message when the result is "unknown", but I'm not sure what the ramifications of removing it are:

https://github.com/cyrusimap/cassandane/blob/e917226a30a84b2991dc23b86215ba029d607429/Cassandane/Unit/TestPlan.pm#L781