airbnb / Showkase

🔦 Showkase is an annotation-processor based Android library that helps you organize, discover, search and visualize Jetpack Compose UI elements
https://medium.com/airbnb-engineering/introducing-showkase-a-library-to-organize-discover-and-visualize-your-jetpack-compose-elements-d5c34ef01095
Apache License 2.0
2.11k stars 107 forks source link

Fix flaky ui-testing GitHub Actions workflow #348

Closed mataku closed 1 year ago

mataku commented 1 year ago

Context

In https://github.com/airbnb/Showkase/pull/342, I made testing GitHub Actions workflow work, which hadn't been working for some time. In the process of code review, I heard ui-testing workflow is unstable.

On my forked repo: https://github.com/mataku/Showkase, I tried to investigate with androidTest result reports but no particularly useful logs were displayed (like: https://github.com/airbnb/Showkase/pull/341#issuecomment-1694632444)

I thought Android emulator itself may be unstable. After updating Android emulator, it seems to be stable. Tried with many retries: https://github.com/mataku/Showkase/actions/runs/5991426769

Changes

vinaygaba commented 1 year ago

@mataku Thank you so much for investigating and helping with making the CI more reliable 🙏🏻

Do you know where the screenshot diffs on this PR are stemming from?

mataku commented 1 year ago

@vinaygaba I don't know the details, but I replaced them after updating emulator because the test failed as it was.


Anyway, as you can see, this change is not enough because workflow results are still flaky. So I will look at it again when I have the time.