microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
62.73k stars 3.38k forks source link

[Docs]: Documentation for `--grep` command line argument gives incorrect string used for matching #30895

Closed marshmn closed 1 week ago

marshmn commented 2 weeks ago

Page(s)

https://playwright.dev/docs/test-cli#reference

Description

The documentation for --grep states this regarding the string that is used for matching: "The regular expression will be tested against the string that consists of the test file name, test.describe titles if any, test title and all test tags, separated by spaces, e.g. my-test.spec.ts my-suite my-test @smoke."

In reality, from my own testing, it appears that the string used to match against also includes the project name.

For example, this fails to match:

--grep "^mytestfile.spec.ts\smy\stest$"

Whereas, this does match:

--grep "^chromium\smytestfile.spec.ts\smy\stest$"

This makes a difference if you are trying to match on the whole string in order to avoid potential sub-string matches.

marshmn commented 2 weeks ago

FYI: this issue came to light through discussion in a Discord thread: https://discord.com/channels/807756831384403968/1241331291425869844

pavelfeldman commented 1 week ago

Would you be interested in sending a PR?

marshmn commented 1 week ago

Yes, sure - will do.