search

americanexpress / jest-image-snapshot

✨ Jest matcher for image comparisons. Most commonly used for visual regression testing.
Apache License 2.0
3.81k stars 197 forks source link

Tests fail because of color difference #283

Closed seamyd closed 1 year ago

seamyd commented 2 years ago

Tests fail because there seems to be a difference in color between base image and test result. I already allow for a 15% difference per pixel but my main problem is that even that doesn't seem enough and I do not understand what causes the difference. The difference also seems to change.

I create my base images locally on a Windows machine and the difference appears when I run the tests on CI which run in a docker container but it uses the same version (jest-image-snapshot "4.2.0").

Did anyone already run into this issue? If additional information is needed please let me know.

Thanks in advance for you time :)

anescobar1991 commented 2 years ago

What I've seen done is running the tests in something like docker locally as well to help with any cross platform differences. That way the tests always run on the same container across windows, macs, linux whether run locally or on CI.

That of course introduces more tooling to your testing chain but it may be worth it for your use case.

Something like this