Closed sergio-sastre closed 2 years ago
Hi @sergio-sastre Thanks for the bug report. I'll look into it and try to get a fix included in the next alpha build.
In the meantime, you might want to try out the on-device high-contrast diff capabilities. You might find it suitable for your needs.
Check out this example https://github.com/Shopify/android-testify/blob/41922e18f3a558a77d8118f10a5a9e73648549c9/Sample/src/androidTest/java/com/shopify/testify/sample/ScreenshotRuleExampleTests.kt#L233-L249
TestifyFeatures.GenerateDiffs visualizes exclusion regions and the exactness tolerance. When enabled, GenerateDiffs will write a companion image for your screenshot test which can help you more easily identify which areas of your test have triggered the screenshot failure. Diff files are only generated for failing tests. The generated file will be created in the same directory as your baseline images. Diff files can be pulled from the device using :screenshotPull.
Hi @sergio-sastre Thanks for the bug report. I'll look into it and try to get a fix included in the next alpha build.
In the meantime, you might want to try out the on-device high-contrast diff capabilities. You might find it suitable for your needs.
Check out this example
TestifyFeatures.GenerateDiffs visualizes exclusion regions and the exactness tolerance. When enabled, GenerateDiffs will write a companion image for your screenshot test which can help you more easily identify which areas of your test have triggered the screenshot failure. Diff files are only generated for failing tests. The generated file will be created in the same directory as your baseline images. Diff files can be pulled from the device using :screenshotPull.
- Black pixels are identical between the baseline and test image
- Grey pixels have been excluded from the comparison
- Yellow pixels are different, but within the Exactness threshold
- Red pixels are different
@DanielJette I'm getting a black diff screen when I run this, is there a way to fix this?
Hmm 🤔
@alaa-alshaikh what is the error/exception being reported by Testify for this test?
Is it possible that you have updated your local baseline to match the failing state, thus causing there to be no differences? Is the baseline image recorded exactly as you desired? You'll want to ensure that the baseline image you have in version control is rendered exactly as you intended. Then, the diff
image will show you any differences with that image. If they're identical, then you'll get 100% black pixels indicating that the current captured image matches the baseline exactly.
@DanielJette Thank you for your answer, so there is a difference between 2 screenshots, I run compare command from the terminal and I got the below image, but not from testify :(
@alaa-alshaikh This could definitely be a bug in the diff algorithm then. Can you provide the following information please?
Desktop:
Target Android Device:
./gradlew testifyKey
]Testify Configuration:
./gradlew testifyVersion
]TestifyFeatures.PixelCopyCapture.setEnabled(true)
]rule.setExactness(value)
]@DanielJette
Desktop:
Target Android Device 1:
Target Android Device 2:
Testify Configuration:
Describe the bug I cannot generate Diff images
This issue relates to:
To Reproduce Steps to reproduce the behavior:
Expected behavior It generates the diff
Desktop (please complete the following information):
Target Android Device (please complete the following information):