search

manykarim / robotframework-doctestlibrary

Robot Framework DocTest library. Simple Automated Visual Document Testing.
Apache License 2.0
48 stars 21 forks source link

Pixel Tolerance and Color Tolerance #47

Open Xorbak86 opened 2 years ago

Xorbak86 commented 2 years ago

What do you think about the possibility to add a pixel tolerance and a color tolerance? Pixel tolerance = Defines the permissible number of different pixels. If the number of different pixels is less than or equal to the pixel tolerance consider the images to be identical. Color Tolerance = Specifies an acceptable color difference at which two pixels should be treated as the same. The color difference is represented as an integer value in the range 0...255 that specifies an acceptable difference for each color component (red, green, and blue) of the compared pixels. Two pixels are considered identical if the difference between the intensities of each of their color components does not exceed the specified value. If ColorTolerance is 0, which is the default, the compared pixels are considered identical only if they are exactly the same color. If ColorTolerance is 255, pixels of each color are considered identical.

manykarim commented 2 years ago

Thank you, those are two intersting points you're bringing up.

  1. Pixel Tolerance: There is already parameter called threshold which we could use easily. It is not the EXACT number of pixels, but a float value between 0.0 and 1.0 (where "0" means, the images need to be EXACTLY the same. while "0.99 means they can be very different) It can be passed as an argument when importing the library, I just noticed a small bug that I need to fix. But afterwards, it could be used as shown below:
*** Settings ***
Library    DocTest.VisualTest    threshold=0.5
Library    Collections

*** Test Cases ***
Compare two images with move tolerance
    Compare Images    testdata/beach_left.jpg    testdata/beach_right.jpg

As an alternative, we could also "return" the actual value of the visual difference. So we could return a value like "0.01" and you would know: There is only a tiny difference between the images

  1. Color Tolerance: Let me think about it a bit more. Currently, the visual comparison is done after conversion to greyscale images. So differences of "brightness" can only be identified - but not the color difference. Will think about a solution.
Xorbak86 commented 2 years ago

Thank you for your quick response and cudos to the project. I tried to implement it by myself, but im lacking python skills. I am happy you take those suggestions into consideration :)

manykarim commented 2 years ago

My Python skills were very poor when I started this project, that's one reason why it's pretty messy and hard to maintain. I already started to do a bit of refactoring, so hopefully it is easier to maintain in the future.

Maybe you can support with writing some tests and preparing some test data, once those two things are implemented?