microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.92k stars 3.67k forks source link

[Feature] Built-in adapter for image comparison library #28578

Open dmtrKovalenko opened 11 months ago

dmtrKovalenko commented 11 months ago

We are doing a lot of visual regression tests during out CI and I noticed that when tests are running without snapshots there is a significant time difference for making and comparing screenshots.

So I build and now use https://github.com/dmtrKovalenko/odiff for my playwright tests. The only addition I'd like to have is the ability to specify the comparison library in playwright instead of completely rewriting the logic of comparing screenshots.

This will make it possible to use the same test-results output, the same scripts for updating snapshots, and do not think about possilbe breaking changes for the while visual regression when updating playwright.

Example

// playwright.config.ts
compareImages: (originalPath: string, actualPath: string, diffPath: string, width, height, options) => { 
   // here is all the logic related to comparing screenshots and outputting the diff
}

Which is by default equals to https://github.com/microsoft/playwright/blob/main/packages/playwright-core/src/third_party/pixelmatch.js

I'd be happy to make a PR if there will be a green light

gskierk commented 3 months ago

How would you like to pass on information about the library? In the configuration file? In the .toHaveScreenshot() method? So we would need to define an interface for adapters first.

dmtrKovalenko commented 3 months ago

Yeah just make an interface and pass an adapter in the configuration