vitalets / playwright-bdd

BDD testing with Playwright runner
https://vitalets.github.io/playwright-bdd/
MIT License
287 stars 33 forks source link

Question:What is the Priority of Tags stated in the playwright BDD run command vs Playwright Config using grep #142

Closed RichardCariven closed 4 months ago

RichardCariven commented 5 months ago

I have my tests split up by Tag into Smoke, Regression, and Dev which I then use different run commands to kick off based on the environment/situation

Eg.

"e2e:smoke": "npx bddgen --tags '@smoke' && npx playwright test --reporter=line",
"e2e:regression": "npx bddgen --tags '@regression' && npx playwright test --reporter=line",

However I now also want to use the Grep functionality in playwright config to set up certain tests to be mobile or web, or both,

{
      grep:[/@web/, /@all/],
      name: "chromium",
      use: { ...devices["Desktop Chrome"] },
    },
    {
      grep:[/@web/,/@all/],
      name: "webkit",
      use: { ...devices["Desktop Safari"] },
    },
    {
      name: "Iphone",
      grep:[/@mobile/,/@all/],
      use: { ...devices["iPhone SE"] },
    },
However, I want to make sure that I still only run my Smoke or Regression tests 

For instance below, I want to be able to only run the Smoke or Regression test, and then only run that against the Mobile device, not have both tests run for every mobile test.

 @smoke
    @mobile 
    Scenario: some smoke test 

    @regression
    @mobile 
    Scenario: some regression test

Does Playwright BDD filter out tests via tags first and then run the playwright.config filter against each test. Or will it just run all the tests that are tagged regardless of if the tag is in the Playwright BDD run command, or the Playwright Config?

is there an alternative way you would suggest setting up these kinds of scenarios?

vitalets commented 5 months ago

Tags passed to bddgen command filter which features/scenarios will be generated. Tags passed to playwright grep filter which of generated tests will run.

For this example:

@smoke
@mobile 
Scenario: some smoke test 

@regression
@mobile 
Scenario: some regression test

Command to run smoke on mobile should be:

npx bddgen --tags '@smoke' && npx playwright test --project=Iphone
vitalets commented 4 months ago

Closing as answered. Feel free to re-open if needed.