search

mkorpela / pabot

Parallel executor for Robot Framework test cases.
https://pabot.org
Apache License 2.0
476 stars 152 forks source link

Wrong ordering with shards used #559

Open patrikwilliam opened 1 year ago

patrikwilliam commented 1 year ago

Pabot has a great feature of shards splitting, but currently does alphabetical order first, splits suites to shards and then orders the suites with ordering file per shard. However, this causes the ordering to be not preserved equally. E. g. ABCD -> 2 nodes, order by ACDB -> [AB],[CD] instead of [AC],[DB]. The more suites you have the worse it is. Unfortunately, this is a significant perf problem since you want to end up running all shards equally, which is currently not possible due to the behavior explained (each suite has different duration). This minor change should do the trick https://github.com/mkorpela/pabot/pull/558, but it is not clear to me why the current behavior is as it is and who might benefit that.

patrikwilliam commented 2 months ago

Would there be anyone looking into this? Thanks!

MattiJarvinen commented 3 weeks ago

The idea is that the shards should execute in the nearly in the same duration. That way all the shards should complete in the fastest combined execution time.

So it picks the shortest test and the longest test into the same pairing.