microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
67.35k stars 3.71k forks source link

[Bug]: test.step() fails in worker fixture teardown #33750

Open vitalets opened 1 week ago

vitalets commented 1 week ago

Version

1.49

Steps to reproduce

If test.step() is called in worker fixture setup - everything works fine. If test.step() is called in worker fixture teardown - there is an error test.step() can only be called from a test.

To reproduce: Please follow instructions in README: https://github.com/vitalets/playwright-issues/tree/test-step-worker-teardown

Expected behavior

Calling test.step() is either allowed in any phase of worker fixture OR is not allowed in worker fixtures.

Actual behavior

Calling test.step() is allowed in setup phase of worker fixture, but fails in teardown phase of worker fixture.

Additional context

No response

Environment

System:
    OS: macOS 13.6.9
    CPU: (8) arm64 Apple M1
    Memory: 58.34 MB / 16.00 GB
  Binaries:
    Node: 20.14.0 - ~/.nvm/versions/node/v20.14.0/bin/node
    Yarn: 1.22.21 - ~/.nvm/versions/node/v20.14.0/bin/yarn
    npm: 10.9.0 - ~/.nvm/versions/node/v20.14.0/bin/npm
    pnpm: 7.27.1 - ~/Library/pnpm/pnpm
  IDEs:
    VSCode: 1.95.3 - /usr/local/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: 1.49.0 => 1.49.0
mxschmitt commented 6 days ago

This is working as expected from our side. We run worker fixture setup as part of a test for convenience reasons and don't do that for worker fixture teardown - hence the error happens. Do you mind elaborating about your use-case? I recommend to not use test.step in worker fixture teardown for now.

vitalets commented 6 days ago

From the user's side the behavior is a bit inconsistent. I can wrap into steps some parts of fixture setup, but can't wrap into steps the fixture teardown. The use-case is trivial - imagine populating a database in a worker fixture:

  myWorkerFixture: [async ({}, use) => {
    await test.step('populate database records', () => { ... }); // <- this is allowed
    await use();
    await test.step('cleanup database records', () => { ... }); // <- this is NOT allowed
  }, { scope: 'worker' }],