takahirom / roborazzi

Make JVM Android integration test visible 🤖📸
https://takahirom.github.io/roborazzi/
Apache License 2.0
654 stars 25 forks source link

Screenshot recreated when updating to 1.10.0 #282

Closed victorpineda-jt closed 3 months ago

victorpineda-jt commented 3 months ago

Hi, I've noticed that after updating the lib version from 1.9.0 to 1.10.0, many screenshots are recreated again. The new screenshots are identical to the previous ones, pixel by pixel (I've used different online tools to compare both images). The resolution is still the same. The only difference I can spot is the file size, which differs by a few bytes. Before recording the images, I ran the verification step, which is working fine and it is not reporting any errors. However, the record command is recreating many (not all) of the previous screenshots.

The issue only happens when updating from 1.9.0 to 1.10.0 and executing the record command. Once the new identical screenshots are generated again, it doesn't happen again. But I wonder whether there is a difference on 1.10.0 that makes the screenshots to be recreated again.

Thanks in advance for your great work!

takahirom commented 3 months ago

@victorpineda-jt Thank you for letting me know! I believe it is due to the introduction of Custom context data for images. This feature adds data to the images' metadata. https://github.com/takahirom/roborazzi/releases/tag/1.10.1

Firstly, we offer a flag for disabling this feature by setting roborazzi.contextdata=false in gradle.properties. Secondly, we can update images by executing ./gradlew verifyAndRecordRoborazziDebug if the image pixels have changed.

I understand that changing the "golden images" is not ideal. We might need to consider adding a new option, something like record.newandpixelchanges=true.

Do you have any thoughts on this?

victorpineda-jt commented 3 months ago

Thank you very much, roborazzi.contextdata=false did the trick. As we are not using metadata for now, it's ok to disable it to prevent golden images to be recreated again.

I guess if we enable it in the future, it won't be a big deal that they are recreated once 👍