pedrovgs / Shot

Screenshot testing library for Android
Apache License 2.0
1.18k stars 116 forks source link

Feature Request / Question - Record Screenshots at Runtime #331

Closed Alton09 closed 1 year ago

Alton09 commented 1 year ago

My team is currently leveraging Shot to perform screenshot tests in Firebase Test Lab. So far it works alright, but we are running into issues where the recorded screenshot fails when compared to the golden screenshot even when the screenshots appear to be identical. I've heard that this issue can arise when the environment used to record the golden screenshots is different than the environment that is used to verify screenshots for tests. We are ensuring that the emulator configuration used is the same, but the only difference is the host machine that runs the tests. i.e. We record golden screenshots on our developers local MacBook Pros and verify the screenshots in Firebase Test Lab which I'm assuming is running a docker instance running Linux.

So, my ask here is if there is a way currently in the library to control whether to record or verify screenshots at runtime without using the gradle commands? We'll need to be able to record screenshots at runtime so that we can ensure both the recording and verifying of screenshots happens both in Firebase Test Lab. The reason we need this functionality is because it is not possible (from my understanding) to pass gradle commands to be executed in Firebase Test Lab. Let me know if more details are needed here. Thank you!

tdrhq commented 1 year ago

@Alton09 For this you really just want to make sure you're always recording screenshots in the same environment that you verify it (i.e. don't record on developer laptops)

Alton09 commented 1 year ago

Thanks! Yeah I realized this a while back as well and forgot to close this issue. Closing now 😄