search

ojkelly / yarn.build

Build 🛠 and Bundle 📦 your local workspaces. Like Bazel, Buck, Pants and Please but for Yarn Berry. Build any language, mix javascript, typescript, golang and more in one polyglot repo. Ship your bundles to AWS Lambda, Docker, or any nodejs runtime.
https://yarn.BUILD
MIT License
326 stars 28 forks source link

[Feat Req] Improvement of testing parallelisation #214

Open Hideman85 opened 2 years ago

Hideman85 commented 2 years ago

Hi there, I see today a big difference between running yarn jest (38s) in my module and the yarn test (2m49s)

My guess is that yarn.build start jest in sequential because it already manage the parallelisation of the modules. This could work really great if you have multiple modules that are independent but when you start to be close to the root module (that use lot of the other one) you start to run test 1 module by 1 module.

I'm wondering here I have 12 cpu threads and 2 modules only can be executed in parallel, could we distribute 6 cores for each runner (parallel jest with 6 workers)?

Please let me know if my assumption is correct, if you think it might be a config mistake or even if that should not be the current output and then is a bug. If you have also in mind a current workaround that would be awesome 🙂

Looking forward to some help

ojkelly commented 2 years ago

I think you might have just hit of the underlying reason why test warrant being its own command. It currently just executes the same strategy as build, which is a topologically optimised dependency order. But, tests really cannot depend on the output of other tests — and so they should just run with as much parallelism as desired.

I don't have a work around at the moment, but I can see this as a huge perf optimisation so I'm going to try and get it done this week.