microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.83k stars 3.67k forks source link

[Feature]: Allow multiple global setup and teardown files in the playwright configuration #32369

Closed Sid200026 closed 3 weeks ago

Sid200026 commented 2 months ago

🚀 Feature Request

Allow globalSetup and globalTeardown properties to take in multiple files as input, thus customers can have multiple global setup and teardown files specified in their playwright configuration.

Example

import { defineConfig } from '@playwright/test';

export default defineConfig({
  globalSetup: [
    './database/global-setup',
    './base/global-setup'
  ],
  globalTeardown: [
   './database/global-teardown',
   './base/global-teardown'
  ]
});

Motivation

  1. A single function for setup and teardown makes it difficult for users to maintain setup/teardown files, where they need to communicate with a lot of services to setup the testing environment - adding mock data to database, signing up fake users for testing, bypassing auth, etc. Multiple files will keep this implementation clean.
  2. In a large project, with multiple playwright test suites, we would like to have a single base playwright config (with global setup and teardown) that will be inherited by all the test suites. Each test suite, can also define their own global setup and teardowns. Currently, it gets overriden and each test suite needs to modify their own setup/teardown files to accomodate the base setup/teardown files. Ideally, we would like to simply add the base setup/teardown files to the setup/teardown array in each configuration.
Skn0tt commented 2 months ago

Today, you can have one globalSetup file that imports all the others:

// project-specific-global-setup.ts
import "./database/global-setup"
import "./base/global-setup"

Does this help solve your pain point? If not, what makes this existing solution suboptimal?

Sid200026 commented 2 months ago
// setup.ts
import setupDatabase from "./database-setup";
import setupBase from "./base-setup";

export default () => {
    setupDatabase();
    setupBase();
}

This is what users have to do currently, create a single setup file and call the respective setup/teardown methods (these methods can come from any type of file, does not need to be a file with a default export).

This presents another problem during teardown. Suppose I have 4 different teardown methods that does independent cleanups. Now the user is responsible for ensuring that if one of the teardown method fails, the others still continue, by adding try catch blocks. Ideally here, since these are independents, if playwright allows customers to pass multiple files, an end user does not need to worry about handling failures so that other tasks can proceed.

  globalTeardown: [
   './database/global-teardown',
   './base/global-teardown',
  './users/global-teardown',
  './infra/global-teardown'
  ]

Secondly, we want to eliminate the pain of developers having to add base setup/teardown files manually inside the test suite's setup/teardown file and worry about ordering of function calls, error handling, etc.