search

microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
67.11k stars 3.69k forks source link

[Docs]: --only-changed CLI flag in .test vs .spec usage #33711

Closed 501NotImplemented closed 17 hours ago

501NotImplemented commented 20 hours ago

Page(s)

Hi. I am trying to adopt the --only-changed CLI flag added in Playwright v.1.46 and initially was not able to make the Playwright running only the one changed test fixture instead of full test scope. As it turned out, the --only-changed detects the tests that have .spec part in the filename, my existing tests were named with the .test part, e.g. login.test.ts, so the CLI flag did not make any difference.

After I've renamed the files, to match the .spec pattern and changed only one test file, the npx playwright test --only-changed command correctly detected the changed file and ran only it.

Is it a behavior by design? I guess so from the test suite https://github.com/microsoft/playwright/blob/main/tests/playwright-test/only-changed.spec.ts

If so, I'd like to ask to add the .test mask support if possible, our current codebase has mix of usage, so it would be nice to support both naming conventions, while we will standartize it on the project convention level. Or explicitly describe the --only-changed mechanism in the CLI reference https://playwright.dev/docs/test-cli#reference

Description

--only-changed works only with test fixtures that use .spec naming convention.

dgozman commented 17 hours ago

@501NotImplemented Thank you for the issue! I have just checked, and --only-changed correctly works with something.test.ts files as well. If you believe there is a bug in --only-changed in Playwright, please file a new issue by following the "Bug Report" template.