search

jessezach / RobotEyes

Image comparison for Robot Framework
MIT License
73 stars 24 forks source link

images differences are captured correctly but robot test is passing even when there is a difference with tolerance set to zero #103

Closed abdalhamidElnaggar closed 1 year ago

abdalhamidElnaggar commented 1 year ago

Describe the bug robot test is passing even when there is difference between two images and tolerance is set to 0

To Reproduce Steps to reproduce the behavior: 1-create new test to compare two images by compare images keyword or compare two images 2-make sure there is a difference between the two images 3-run the test and check robot result

Expected behavior robot test should fail as there is a difference between the two images Screenshots If applicable, add screenshots to help explain your problem. attached the two images and the diff one Desktop (please complete the following information):

Additional context robot_test home_ref home_actual home_diff

abdalhamidElnaggar commented 1 year ago

it seems like magick is returning the difference as 0 , can this be an issue with magick ?

jessezach commented 1 year ago

@abdalhamidElnaggar I took a look at this. Magick returns a value of 0.002 which is a difference of 0.2%. This gets casted to int which is converting the difference to 0. I'll fix the code to retain the exact value instead of doing the int conversion.

Meanwhile a workaround for this could be to use capture element keyword and capture a portion of the page (headers for example). This should return a higher difference value for the mismatch and would result in a failure

abdalhamidElnaggar commented 1 year ago

@jz-jess thank you .. i will wait for the fix as my plan is to compare full pages not just elements. thanks again much appreciated.