[Feature] Expose root `Suite` to the global setup

[xv-andre-s]

Since Playwright 1.19, the list of cases is now generated before the Global Setup (see https://github.com/microsoft/playwright/issues/12018 for more details).

It would be great if we could expose root Suite to the global setup - containing exactly the tests to be run.

This could be beneficial for use cases where we would like to run certain pre-conditions during global setup depending on the list of cases to be executed. For example: authentication setup being carried out only when cases that require it are slotted to be executed, otherwise do not authenticate.

[xv-andre-s]

Wondering what is the team's take on this one? I see it had the version labels removed a while back, so it's not clear whether this is still in slot to be implemented anytime in the near future.

IMHO, this would be a valuable feature 👍

[aoj2]

@dgozman, curious to know if this is something that you are planning to add at some point? I was using a custom test reporter as workaround to get hold of the running tests & store them in env. var to then be used in global setup, but this stopped working from version 1.31.0 it does not work any longer, seems like global setup is now running before the reporter onBegin method. 😞

class TestRunReporter implements Reporter {
  onBegin(config: FullConfig, suite: Suite) {
    const allTests = suite.allTests();
    // Set tests env. var to be used in global-setup
  }
}

[dgozman]

@aoj2 Indeed, this has been changed in v1.31. Take a look at dependencies feature instead. It allows you to have different dependencies for each project, and so you can have separate setups that would only run when needed.

[DudaGod]

Are there any plans to implement this feature? In our project, we want to validate the uniqueness of the test name without linking to the file in which the test is located. We collect test execution statistics by their full name and don't want to lose it when developer move a test from one file to another. So we want implement validation on global setup section.

If this task is not in the nearest plans, can I implement it myself? Maybe there are some pitfalls that I should consider when implementing this feature?

[DudaGod]

I found that I can change the order in which the methods are called here in order to read tests before launch global setup. And send root suite here as second argument.

But similar code is also called in UI mode here. And changing the call order in it is a little more difficult.

So before implementation I want to know what the maintainers think about it.

cc @pavelfeldman

[DudaGod]

@dgozman @aslushnikov can someone talk with me?

[dgozman]

@DudaGod

So we want implement validation on global setup section.

Can you implement this in Reporter.onBegin() instead? We are hesitant exposing Suite to the global setup, because it fixes the order of reading files vs running global setup in a way we don't want to commit to. Therefore, we'd be happy to find another solution for your needs.

[DudaGod]

Can you implement this in Reporter.onBegin() instead?

Yes, I can implement this on onBegin but throwing an error does not interrupt the tests execution. You suggest ending the process with process.exit(1)? In my opinion not the best solution.

[dgozman]

@DudaGod

Yes, I can implement this on onBegin but throwing an error does not interrupt the tests execution.

I see. We do not think this is something a reporter should do. Sorry for misleading you.

---

So I'd recommend running two passes of npx playwright test:

• Run npx playwright test --list and analyze the list of tests to your liking.
• If everything's ok, run npx playwright test.

Overall, a layered approach looks like the most flexible one. Let me know what you think.

[DudaGod]

So I'd recommend running two passes of npx playwright test:

Run npx playwright test --list and analyze the list of tests to your liking. If everything's ok, run npx playwright test. Overall, a layered approach looks like the most flexible one. Let me know what you think.

I thought about this solution, but I don’t really like it because the tests will be read 2 times and waste extra time. In my small project with 30 tests command npx playwright test --list runs 2 seconds.

[asanto-hextrust]

Any updates on this issue? 🙏

[siurbele420]

I strongly believe that incorporating this feature would significantly enhance the project's flexibility. In my specific use case, being able to read the test names in the queue after initiating the Playwright test command would allow me to make informed decisions on whether or not to launch the local server. This is crucial because, in my test suite, tests are tagged with different labels, and not all of them require the server to be running. By avoiding unnecessary server setups, which can be time-consuming, this feature would streamline the testing process, making it more efficient based on our needs.

[ruubytes]

Here to bump this issue, because having the Suite exposed in the global setup makes it, for example, a lot easier to orchestrate the spawning of appium servers for android emulator tests.