mozilla-mobile / reference-browser

A full-featured browser reference implementation using Mozilla Android Components.
Mozilla Public License 2.0
576 stars 162 forks source link

Intermittent UI test failure - AddonsTest.removeAddonTest #1710

Open sv-ohorvath opened 2 years ago

sv-ohorvath commented 2 years ago

Firebase Test Run:

Flaky 1/2 tries. https://console.firebase.google.com/project/moz-reference-browser-230023/testlab/histories/bh.b4e77beaed81bc1c/matrices/7623436030602708407/executions/bs.e0c7fc91709e535c

Stacktrace:

java.lang.AssertionError at org.junit.Assert.fail(Assert.java:87) at org.junit.Assert.assertTrue(Assert.java:42) at org.junit.Assert.assertNotNull(Assert.java:713) at org.junit.Assert.assertNotNull(Assert.java:723) at org.mozilla.reference.browser.ext.UiDeviceKt.waitAndInteract(UiDevice.kt:20) at org.mozilla.reference.browser.ui.robots.AddonsManagerRobot.dismissAddonDownloadCompletedPrompt(AddonsManagerRobot.kt:60) at org.mozilla.reference.browser.ui.AddonsTest$removeAddonTest$3.invoke(AddonsTest.kt:98) at org.mozilla.reference.browser.ui.AddonsTest$removeAddonTest$3.invoke(AddonsTest.kt:93) at org.mozilla.reference.browser.ui.robots.ThreeDotMenuRobot$Transition.openAddonsManager(ThreeDotMenuRobot.kt:136) at org.mozilla.reference.browser.ui.AddonsTest.removeAddonTest(AddonsTest.kt:93)

Build:

debug 11/4. Started with #1704

AndiAJ commented 2 years ago

Flaky 1x on 11/18 Main Firebase link

sv-ohorvath commented 2 years ago

Now it hit a crash, 1/2 tries: https://console.firebase.google.com/project/moz-reference-browser-230023/testlab/histories/bh.b4e77beaed81bc1c/matrices/6687367008273650919/executions/bs.885ef2dbeea6fd82/issues

java.lang.IllegalStateException: Couldn't find the primary search account among the accounts. FATAL EXCEPTION: BG Thread #3 Process: com.google.android.googlequicksearchbox:search, PID: 16220 java.lang.IllegalStateException: Couldn't find the primary search account among the accounts. at com.google.android.libraries.search.a.c.b.m.apply(PG:2) at com.google.apps.tiktok.tracing.dw.apply(PG:2) at com.google.common.w.a.g.d(PG:2) at com.google.common.w.a.h.run(PG:9) at com.google.android.libraries.i.ah.run(PG:1) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at com.google.android.libraries.i.e.run(PG:2) at com.google.android.libraries.i.s.run(PG:4) at java.lang.Thread.run(Thread.java:919)

Posted a question on Firebase slack channel if this might be a device-related crash: https://firebase-community.slack.com/archives/C1MTSQ5QT/p1638800721298500 LE: in response to my question on Firebase slack: "com.google.android.googlequicksearchbox is a package built into our virtual devices, and we can't remove/disable it easily. It's also integrated into the OS, so it may not be possible at all. I can't tell what the root cause of this error is, but it can be possible that a recent change in the googlequicksearchbox is causing these issues. "

sv-ohorvath commented 2 years ago

Flaky on 1/24: https://console.firebase.google.com/project/moz-reference-browser-230023/testlab/histories/bh.b4e77beaed81bc1c/matrices/5049057674011791322/executions/bs.4385fba2166fd047/testcases/1

AndiAJ commented 2 years ago

Flaky on 2/17 Main Firebase link

Archaeopteryx commented 2 years ago

Today: 2/2 failures, rerun with a new task succeeded:

androidx.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (with id: org.mozilla.reference.browser.debug:id/allow_button and with string from resource id: <2131820794>) 
sv-ohorvath commented 2 years ago

From the logs, it seems that it timed out waiting for a confirmation dialog to install an add-on. Similar failure on https://github.com/mozilla-mobile/fenix/issues/24815

AndiAJ commented 1 year ago

Closing, as it didn't fail recently.

AndiAJ commented 1 year ago

► Failed 1x on 9/14 Firebase link Firebase link

The problem seems to be happening after clicking the "Ok got it" button, the icon of uBlock Origin disappears, and another confirmation dialog is displayed.

AndiAJ commented 11 months ago

Failed 1x on 10/12 after Update Android Components version to 120.0.20231011040555. Firebase link Firebase link

AndiAJ commented 3 months ago

► 6/5 Firebase link