Closed shaker-project closed 4 years ago
Can you please explain what fails in the above test testNavigation
, since it does not have any assertions?
We apologize for the delay in responding.
Well, the exception raised was NoMatchingViewException
.
We investigated that, the test fails on line 82 of the selectMenuItem
method when it is called selectMenuItem(SETTINGS)
on line 59 of the Navigation
class. We believe that this happens because the rendering of the element can be slow, therefore, making Espresso not find the element.
The file: output_wifi_testNavigation.txt, contains the test execution outuput.
We run your test cases for 50 times and found that the following test is flaky [1] (i.e., they pass or fail non-deterministically):
We ran these tests on a generic Android Emulator (AVD) with Android API version 28.
To reproduce failures on these tests, please run the following script on gist: https://gist.github.com/shaker-project/aa36af5d1be2638caf8f6e2e50768864 If you didn't understand any point of the script we can explain.
We require that stress-ng is installed. If not, run $> sudo apt-get install stress-ng.
If you confirm this behavior, our suggestion is to add the annotation @TestFlaky on these tests. See https://developer.android.com/reference/androidx/test/filters/FlakyTest
If you wish for a repair proposal, please let us know.
[1] https://hackernoon.com/flaky-tests-a-war-that-never-ends-9aa32fdef359