[Bug]: testInfo.config.grep does not correspond to the CLI grep argument

[avaeroh]

Version

1.44.0

Steps to reproduce

The grep CLI arg does not correspond to the runtime testInfo.config.grep value. Example repo: https://github.com/avaeroh/playwright-grep-config-bug.

Expected behavior

testInfo.config.grep is a string or regex reflecting the CLI arg passed in with --grep or -g.

Actual behavior

greps value is always /.*/

Additional context

No response

Environment

System:
    OS: macOS 14.5
    CPU: (10) arm64 Apple M1 Pro
    Memory: 105.33 MB / 16.00 GB
  Binaries:
    Node: 18.19.0 - ~/.nvm/versions/node/v18.19.0/bin/node
    npm: 10.5.1 - ~/.nvm/versions/node/v18.19.0/bin/npm
    pnpm: 8.15.6 - ~/.nvm/versions/node/v18.19.0/bin/pnpm
  IDEs:
    VSCode: 1.89.1 - /usr/local/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.44.1 => 1.44.1

[dgozman]

@avaeroh This is not as straight-forward as we would like it to be.

For example, consider a config file:

export default {
  grep: /foo/,
}

If you run npx playwright test --grep=bar, what do you expect to see in testInfo.config.grep? Note that Playwright will only run tests that match both /foo/ and /bar/ today.

[avaeroh]

@dgozman In the instance a CLI arg is provided I personally would expect that the testInfo.config.grep represents the overwrite. Could the static & runtime config not be separated into independent objects for the benefit of reflection, e.g. could a testInfo.runtimeConfig equivalent be made available?

I currently have a scenario where some tests tagged @auth use the [auth.setup.ts](https://playwright.dev/docs/auth#basic-shared-account-in-all-tests) pre-test hook. If I look to execute tests that do not have this tag, the auth step still executes. The only way to avoid this currently is to run auth & non-auth tests through a separate 'project', but this will make things needlessly complex when projects already reflect a multitude of browsers/devices, whereas test tags that are already responsible for test filtering could cleanly add a skip on the auth setup, e.g.:

setup("authenticate as auth user", async ({}, testInfo) => {
    test.skip(!testInfo.config.grep.includes("@auth"));
...
});