cashapp / paparazzi

Render your Android screens without a physical device or emulator
https://cashapp.github.io/paparazzi/
Apache License 2.0
2.31k stars 215 forks source link

Golden images in `/build/paparazzi/failures` are not stored in some error cases #1716

Open SimonMarquis opened 10 hours ago

SimonMarquis commented 10 hours ago

I'm not sure if this should be classified as a bug of a feature request:

Description Golden images are stored in /build/paparazzi/failures when there is a regression (next to the delta-*.png files) but, they are not when:

Steps to Reproduce I can create a simple project for that if it is really necessary, but I think the description is fairly simple, and easy to reproduce, without special setup. Please let me know.

Expected behavior Having golden screenshots generated even when the test fail because it does not match the reference (or if the reference is missing) would be very valuable in CI scenarios, where the newly created golden images could be immediately extracted without having to run an entire pipeline with a new execution of recordPaparazzi Gradle task. Also, running the recordPaparazzi might lead to unrelated golden images being updated (with invisible variations) that would flood the review diff.

Additional information:

Note: In previous versions (pre-OffByTwo), when there was a regression with different image sizes, the delta and golden images were stored to disk on /out/failures.