search

atata-framework / atata

C#/.NET test automation framework for web
https://atata.io
Apache License 2.0
489 stars 80 forks source link

Atata Screenshot Comparison #334

Open iarif20-devops opened 4 years ago

iarif20-devops commented 4 years ago

Hi,

I am working on a test to match two screenshots. I have added some custom functionality to compare the screenshot taken by a user with a screenshot taken using Atata framework.

For some reason, the images comparison don't pass. However, if I generate both images via Atata, the comparison passes.

In all cases, I make sure in my algorithm that images are of the same size. Also, the image Bit Depth is same.

Do you have any idea, what could be cause here.

Apologies for not passing any sample data. It is just because of the privacy.

However, should you require, I will prepare some.

Thanks in advance.

Regards,

Muhammad

YevgeniyShunevych commented 4 years ago

Hey,

It's hard to say at this point. First, I assume that you save screenshots in PNG format, not JPEG. I think you should check with your comparison algorithm the difference between images, how much are they different. Generate a delta image with highlighted pixels (red or pink color) that differ. That should make it more clear to understand the difference.

Also, it is absolutely OK, from my point of view, to get the expected screenshot in an automatic way from Atata/WebDriver. Run the test once to get the expected screenshot and then store it somewhere for the future comparison as expected image.

Also, I can advise taking a look at Applitools, which does the screenshot comparison in a good way. But it is not free.

iarif20-devops commented 4 years ago

Thanks for your feedback. I'll look to see if I can stick with Atata generating testing samples for me.

Piedone commented 2 years ago

We implemented this in the Lombiq UI Testing Toolbox that uses Atata. Check it out here: https://github.com/Lombiq/UI-Testing-Toolbox/issues/85