microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
65.51k stars 3.57k forks source link

[Feature] Allow to specify ignoreSnapshots, updateSnapshots setting per project #26729

Open blessanm86 opened 1 year ago

blessanm86 commented 1 year ago

Currently, we have configured our playwright as 2 projects - local and ci. The CI project just runs the tests in a remote moon instance. We would like to do screenshot testing only on the moon instance for stability. So the flow would be, that visual comparison only runs on CI while users can update the reference screenshots locally.

Currently, we can only specify the ignoreSnapshots, updateSnapshots setting common for all projects. It would be great if we could specify at a project level

{
  name: 'local',
  ignoreSnapshots: true,
},
{
  name: 'CI',
  ignoreSnapshots: false,
  updateSnapshots: 'all',
  use: {
    connectOptions: {
      wsEndpoint: 'wss://moon.tools.xxx/playwright',
    }
  }
}
yury-s commented 1 year ago

You can configure ingnoreSnapshot option as in our example:

export default defineConfig({
  ignoreSnapshots: !process.env.CI,
});

And then run the test locally with CI=1 when one needs to update snapshots:

CI=1 npx playwright test -u

Would that work?

blessanm86 commented 1 year ago

That's what we are doing now. Just wondered why these options couldn't also be project specific.

We use a nx monorepo where we have a common playwright config that is exported from a shared lib.

Project level config would allow us to just make the change in the lib. But now have to tell all the users to prefix CI=true in their nx project.json target