FormidableLabs / react-native-owl

Visual regression testing library for React Native that enables developers to introduce visual regression tests to their apps.
https://formidable.com/open-source/react-native-owl/
MIT License
639 stars 28 forks source link

[Feature Request] Enable tolerance threshold settings for screenshots #140

Closed muhammadkhan0009 closed 1 year ago

muhammadkhan0009 commented 1 year ago

Hello everyone, First of all, thank you very much for doing such a great work on this library. We are the early adopter of this library and really excited about its future because of its help to make sure that our app UI works as expected.

Problem :

We have recently noticed that our test cases are failing due to minor change in the screen pixels which leads us to further explore the codebase of your library. The more details are given in the screenshot.

Proposed solution :

While exploring the codebase we have found out that toMatchBaseline function only accepts the latestPath param to compare baseline image with latest images by using pixelmatch lib under the hood. Upon further investigation we have found that the pixelmatch also accepts threshold parameter as an argument to expose some tolerance.

pixelmatch(img1.data, img2.data, diff.data, width, height, {threshold: 0.1});

I think it would be nice to enable toMatchBaseline function to accommodate additional parameter as options where users can add the tolerance threshold so that tests don't get failed unnecessary i.e.

toMatchBaseline(latestPath:string, options?:{threshold?:number} )

Screenshot :

Screenshot 2023-02-17 at 14 27 01

Zoomed in Screenshot :

Screenshot 2023-02-17 at 14 36 38

@manosim As you can in the above screenshots that test is failing due to very minor pixel difference (i.e. red colour in the screenshot) . I think library should be tolerant enough to deal with these type of issues.