Closed garciallorente-fer closed 3 weeks ago
mxschmitt
commented
4 months ago Hey! Could you help bringing this issue down to a more real-world scenario / code which we can run locally? I tried to reproduce the behaviour which you are mentioning but was not able to.
This is the repository I've created: https://github.com/mxschmitt/playwight-issue-30296
Thanks!
garciallorente-fer
commented
4 months ago Thanks for the reply @mxschmitt , I have created a PR for you with a more real-world scenario / code which we you run locally npm test
https://github.com/mxschmitt/playwight-issue-30296/pull/1
In this example we can see a 50% increase in time inefficiency
Project1: login test project (2s) Project2: e2e-auth project (5s) > requires project1 Project3: e2e-nonAuth project (10s) > no dependencies
Real world scenario , it should take just 10s to run in total (project3 time) But it takes 15s (project2+project3 times)
pavelfeldman
commented
3 weeks ago Thank you for your contribution to our project. This issue has been closed due to its limited upvotes and recent activity, and insufficient feedback for us to effectively act upon. Our priority is to focus on bugs that reflect higher user engagement and have actionable feedback, to ensure our bug database stays manageable.
Should you feel this closure was in error, please create a new issue and reference this one. We're open to revisiting it given increased support or additional clarity. Your understanding and cooperation are greatly appreciated.
Version
1.43.0
Steps to reproduce
I have a big time increase in my real-life project because of the way Playwright runs projects & projects dependencies. It currently runs projects in phases, each phase contains projects that are free to go and waits until all of them are finished, but that waiting causes big total time increases.
This is a close example to my real-case scenario:
projects: [ { name: 'End2End-NonAuth' , testMatch: '**.nonAuth.**' }, { name: 'LOGIN-Test-setup', testMatch: '**login.setup.ts**' }, { name: 'End2End-Auth', testMatch: '**.e2e.**', use: { storageState: loggedInStatePath }, dependencies: ['LOGIN-TEST-SETUP'] } ]As you can see here, Project 'End2End-NonAuth' can run all the tests at the same time as 'LOGIN-Test-setup'.Project "End2End-NonAuth" has like 10 tests that take up to 60 seconds total time (because they run in parallel that is the max time one of them test takes) they do take their time because need to be end-to-end tests, like most people with have in their projects.
Project "LOGIN-Test-Setup" has JUST ONE test file, because this is test is the dependency a lot of tests have, to login first before running some end-to-end tests, as most of people would have in their project configurations. (it just takes like 20 seconds)
If I am using 50 workers (Playwright Testing) , when you try to run all these tests, this is the sequence:
This time just increases x3 times more with retries=2 , if one of the test fails, delaying massively the detection of a test error, which is what we want when something is breaking and the most important time efficiency we want to see.
Expected behavior
SOLUTION: Every phase should not wait for all the phase projects to finish, it should only wait until the dependency project is finished
All the workers are always working 100% of the time, with no time loss, as it only waits for the dependency project to finish,, instead of just waiting for the project that has the dependency to finish.
Actual behavior
Playwright currently runs projects in phases, each phase contains projects that are free to go and waits until all of them are finished.
There are workers free to use for other phases that are not being used, while waiting for the whole phase to finish, instead of just waiting for the project that has the dependency to finish. That can create a lot of total time increase.
Additional context
For more context please look at the previous bug that was not solved but still closed: https://github.com/microsoft/playwright/issues/29714
Environment