microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.41k stars 3.63k forks source link

[BUG] teardown project is run before the tests #29068

Closed dorfsmay closed 9 months ago

dorfsmay commented 9 months ago

System info

Source code

playwright.config.ts is the one generated by npx playwright init, with this modification where I add a global setup and teardown, and the dependencies:

projects: [
  {   
    name: 'setup',
    testMatch: /global\.setup\.ts/,
    teardown: 'teardown',
  },
  {   
    name: 'teardown',
    testMatch: /global\.teardown\.ts/,
  },  
  {
    name: 'chromium',
    use: { ...devices['Desktop Chrome'] },
    dependencies: ['setup'],
  },  

  {   
    name: 'firefox',
    use: { ...devices['Desktop Firefox'] },
    dependencies: ['setup'],
  },

The tests are the ones generated by init. tests/global.setup.ts:

console.log("H E L L O !")

tests/global.teardown.ts:

console.log("B Y E")

Steps

Expected

H E L L O !

Running 2 tests using 2 workers
  2 passed (5.3s)

B Y E

Actual

H E L L O !
B Y E

Running 2 tests using 2 workers
  2 passed (5.3s)

Additional notes This is not just a console output/buffer issue. This is minimum example to test the issue, but in the original work, the global setup project use an API to add users and the global teardown project use the same API to delete those users. When looking at the backend log, I can see that the users are added, the users are then removed, and finally the login attempts are made, but fail since the users have now been deleted.

Is there a way to trace the project priorities to try to understand why teardown is run before the tests?

dorfsmay commented 9 months ago

This is solved, I added using test() in my setup and teardown and it works. My new tests/global.teardown.ts:

import { test } from '@playwright/test'

test('test A', async ({ page }) => {
console.log("B Y E")
})

I wish this was mentioned in the documentation.

mxschmitt commented 9 months ago

Playwright has two ways of doing setup/teardown see here: https://playwright.dev/docs/test-global-setup-teardown

In your case you are doing option 1, which is the new / preferred way of doing it. If you have any concrete docs suggestions, let us know!

dorfsmay commented 9 months ago

In the doc, I would:

psikri commented 6 months ago

Hi,

I tried following these steps, but for me playwright is still executing teardown first and test methods later, I am just trying to execute some of the api test cases, so there is no UI config, here is my - config file

`

import { PlaywrightTestConfig } from '@playwright/test'

const config: PlaywrightTestConfig = { timeout: 10 500 1000, expect: { timeout: 10000, }, workers: 10, retries: 1, testDir: '../tests/RegressionTests/ApiTests/', reporter: [ [ 'monocart-reporter', { name: ' API Test Report', outputFile: './test-results/BffApi_report.html', }, ], [ 'junit', { name: ' API Test Report', outputFile: '../test-results/Api_report.xml', }, ], ], use: { trace: 'retain-on-failure', }, globalSetup: '../environment/envSetup.ts',

projects: [ { name: 'setup', testMatch: /global.setup.ts/, teardown: 'teardown', }, { name: 'teardown', testMatch: /global.teardown.ts/, }, { name: 'API', dependencies: ['setup'], }, ], }

export default config

` And on execution the output is

image

Given below is teardown file -global.teardown.ts

`

import { test } from '@playwright/test'

test('Disconnect DB', async () => { console.log('Disconnecting DB') //CODE TO DISCONNECT TO DB })

`

Setup file- global.setup.ts code- `

import { test } from '@playwright/test'

test('Start API Test execution', async () => { console.log('API Test case execution starts after this') })

` Environment Details

"@playwright/test": "^1.42.1" "playwright": "^1.42.1", OS- Windows 10 Enterprise Node Js- v18.12.1

Can you help me to know what I am missing here, I could not find any documentation to help me out on this, so posting it here