codeceptjs run-workers X can do with sequencing improvements

sethvanwykJET commented 8 months ago

What are you trying to achieve?

codeceptjs run-workers 5 should be more performant and pull tests from a pool instead of having each test pre-assigned to a thread

What do you get instead?

Currently run-workers will create a pool of all tests when codeceptjs run-workers 5 is executed.

It then pre-assigns each test to a CPU thread and starts running all tests assigned to each thread.

This creates an even distribution of tests per thread. The problem is that not all tests are equal in terms of runtime

Small sample from end of test execution

[01]   ✔ in 4354ms
[04]   ✔ in 4794ms
[02]   ✔ in 3328ms
[04]   ✔ in 7131ms
[03]   ✔ in 13437ms
[01]   ✔ in 11558ms
[05]   ✔ in 21881ms
[02]   ✔ in 7051ms
[02]   ✔ in 4259ms
[03]   ✔ in 15055ms
[02]   ✔ in 5372ms
[03]   ✔ in 5274ms
[03]   ✔ in 17304ms

If all tests in thread 1 would complete in 2 minutes, then thread 1 would not pick up any more tests even if thread 2 - 5 have 6 tests all still waiting in their queue. This means that tests are only as fast as the slowest thread rather than allowing tests to be picked up by an idle thread.

In the example above thread 02 and 03 were slower than others. thread 01, 04 and 05 were waiting in idle while 02 and 03 were continuing with their queue.

In an ideal world, thread 01, 04 or 05 would see this incomplete queue and pick up a task to speed up test performance.

Details

CodeceptJS version: 3.5.3
NodeJS Version: 18
Operating System: MacOS / Linux
puppeteer

DavertMik commented 7 months ago

I agree that this implementation would be better.

I think our implementation was created based on the API of mocha which didn't have any parallelization on the time we implemented workers. I think there might be a better way to pull tests and push them to different workers but I don't think I have enough time to implement this.

However, if you know how to do this you are welcome to send PR with implementation!

sethvanwykJET commented 7 months ago

I'm not too familiar with this myself and uncertain when I'll have the time to look into it

If I get a moment I might investigate, but I wouldn't expect much 😅

github-actions[bot] commented 4 months ago

This issue is stale because it has been open for 90 days with no activity.

Horsty80 commented 3 months ago

This will be a very usefull improvment for run-workers command It's frustrating to have run-workers 8 and at the end see only 1 or 2 worker running the last 10/20 test x) where are the 6 other worker ? I think they take a break with a cup of coffee x)

github-actions[bot] commented 2 weeks ago

This issue is stale because it has been open for 90 days with no activity.

codeceptjs / CodeceptJS