Shopify / android-testify

Add screenshots to your Android tests
https://testify.dev
MIT License
231 stars 23 forks source link

Screenshot taken in Firebase is different from the local recording #248

Closed Angelia-Gong closed 2 years ago

Angelia-Gong commented 2 years ago

Describe the bug Screenshot tests keep failing because of image comparison failure. The "expected image" I recorded has the following emulator configs:

On firebase:

                    gcloud firebase test android run \
                      --type instrumentation \
                      --use-orchestrator \
                      --app app/build/outputs/apk/internal/debug/app-internal-debug.apk \
                      --test FaireUI/build/outputs/apk/androidTest/debug/FaireUI-debug-androidTest.apk \
                      --device=model=flame,version=29,locale=en_US \
                      --environment-variables useSdCard=true \
                      --directories-to-pull ${SCREENSHOT_PATH}
The screenshot taken is different, notice the width of the black bar at the bottom and grey bar at the top: (the images are the same size and dimension, it just looks different in this PR description) Firebase x86_64 x86
firebase_screenshot SampleScreenshotTest_default SampleScreenshotTest_x86

Can't really figure out what's causing this difference

Angelia-Gong commented 2 years ago

Similar ticket, more into in https://github.com/Shopify/android-testify/issues/247