microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
62.73k stars 3.38k forks source link

[Bug]: Having multiple `Page`s on the same domain, JS coverage is only reported for the one that got `stopJSCoverage` called before others #30876

Open andrewbents opened 2 weeks ago

andrewbents commented 2 weeks ago

Version

1.44.0

Steps to reproduce

Example steps:

  1. Clone my repo at https://github.com/andrewbents/playwright-coverage-issue-example
  2. npm install
  3. npm run test

Expected behavior

I expect coverage reports from both Pages to contain files.

Actual behavior

Only the Page that had it's stopJSCoverage method called first has files in the report. The second page reports empty list. If I reverse the order (for (const page of context.pages().reverse()), then the Page that used to report empty list will have files in the coverage, and the one that previously did have the files will report empty list, confirming the guess. Note that the pages have different set of files in their reports.

Additional context

Seems like the issue happens only for pages on the same domain. You can unskip the second test to check.

Environment

System:
    OS: macOS 14.4.1
    CPU: (12) arm64 Apple M2 Pro
    Memory: 226.52 MB / 16.00 GB
  Binaries:
    Node: 18.19.0 - /usr/local/bin/node
    Yarn: 1.22.21 - /opt/homebrew/bin/yarn
    npm: 10.2.3 - /usr/local/bin/npm
    bun: 1.0.33 - ~/.bun/bin/bun
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.44.0 => 1.44.0