Closed DanielJette closed 2 years ago
I'm currently trying this locally:
system_image="system-images;android-29;google_apis;x86_64"
device="pixel_4"
and sending this flag to the firebase gloud
command on CI: --device=model=flame,version=29,locale=en_US
That's yielding slightly different images:
Local | Firebase |
---|---|
![]() |
![]() |
cc @Angelia-Gong
In order to run the screenshot tests successfully, both locally and on Firebase CI, it is important that you maintain an emulator specifically designed for screenshot testing. The Testify Screenshot library requires that you use an identically configured Emulator AVD locally and on CI.
⚠️ Note that Android Studio creates the Pixel 2 with the incorrect DPI. To fix this, launch the emulator and run adb shell wm density 441
from your terminal.
Once the emulator is booted:
adb shell wm density 441
from your terminalWindow animation scale
, Transition animation scale
, and Animator duration scale
to Off
You only need to run these commands once.
On Firebase, you want to use the following configuration:
Pixel2,29,en_US,portrait
"form": "VIRTUAL"
Set the following environment variables on Firebase:
TESTIFY_USE_SDCARD=true
useSdCard=true
Failed screenshots will be posted to:
/storage/emulated/0/Android/data/<<your.package.name>>/files/testify_images/
Another tip:
Run the following command locally on your emulator: ./gradlew app:testifyKey
Then, look in the Firebase logs for the tag Baseline could not be found in <<key>>
.
Check that the testifyKey matches what's being output in the logs.
@Angelia-Gong let me know if this helps/doesn't help. I've found this configuration I posted here works best with Firebase and is a good mid-range device with a reasonable screen resolution. Plus, it's one of the virtual (cheaper) devices on Firebase.
@DanielJette Hi Dan, I have followed the configurations above and the images still look slightly different (notice the colour difference of the top bar. One is greyer and one is darker). At least that's the only difference to my eyes | local | firebase |
---|---|---|
![]() |
![]() |
@Angelia-Gong 🤔 I haven't seen this particular issue before.
My guess is that it looks like some different resources are loading locally compared to on Firebase.
Do you have any alternative resource qualifiers on your project? like -hdpi
or -night
. Is it possible that Firebase is running with a dark/night theme enabled? (Or perhaps running with dark theme locally?)
Is this a new project based on the recent Android Studio template projects? The most recent templates auto-create a themes.xml
in the values-night
directory. So, maybe try removing the night theme?
@DanielJette No we don't have night mode. It's not a very recent project and there's no themes.xml
in values-night
. Would it be related to the virtual GPU Google uses?
@DanielJette No we don't have night mode. It's not a very recent project and there's no
themes.xml
invalues-night
. Would it be related to the virtual GPU Google uses?
@Angelia-Gong I've not encountered this problem myself, but I could certainly see this as possible. If there's alpha blending going on with the colour being used by the status bar/toolbar then this could happen. Are you using a non-opaque colour for either of those elements?
One suggestion you may want to consider is to see if you can get the Sample
app from this repository running on your CI environment. Whether that's successful or not could indicate whether there's an issue in the Testify library, Firebase or the app code itself.
Also, if you wanted to share the code for the test and the Hello world activity you're trying, I could run it on my environment and see if I get the same results.
What's the recommended configuration to get Testify working on Firebase Cloud Test Lab?