CI fails on master branch

[EwoutH]

The test-android job on Circle CI has been failing since the 8th of August on the master branch. 5732d0e was the last commit where it passed, since 31337b5 it is failing.

Blocks #198 and #199.

[diegosanchezr]

Hey @EwoutH, sorry for the delay tackling this issue.

There seems to be a problem with robolectric that I'm looking into. It's being a little bit more difficult to solve than I thought but hopefully I'll be able to fix it this week.

[EwoutH]

@diegosanchezr No problem, thanks for looking into it! I gave it a shot myself today (#202), it was also more complicated than I thought.

PRs shouldn't be merged in the master branch if the CI is not passing, however.

[EwoutH]

@diegosanchezr Have you made any progress so far?

[diegosanchezr]

Not yet unfortunately :( I went through https://github.com/robolectric/robolectric/issues/5235 without success so far

[diegosanchezr]

I made some progress on https://github.com/facebookincubator/spectrum/pull/205. Now there aren't build failures, but now ./gradlew connectedAndroidTest step is failing with "No tests found" error. It seems the test app is crashing but I haven't been able to figure out why yet.

[EwoutH]

Thanks for keeping working on this!

[diegosanchezr]

I tried a few more things and now it seems that ./gradlew connectedAndroidTest works locally on x86_64/v29 emulator.

I tried using that emulator on CircleCI but it's not possible as x86/x86_64 require HW acceleration and they don't support that. Most up to date emulator for arm is using API v25. I haven't been able to make it work there either locally or on CircleCI (getting undefined classes java.nio.file.FileSystemException).

Help is still pretty much appreciated

[EwoutH]

Made some progress in #210. The CI now fails a lot further in the job, on:

com.android.builder.testing.ConnectedDevice > No tests found.[testAVD(AVD) - 4.4.2] FAILED 
No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @Test annotations).
08:32:51 I/XmlResultReporter: XML test result file generated at /root/react-native/android/spectrumtests/build/outputs/androidTest-results/connected/TEST-testAVD(AVD) - 4.4.2-spectrumtests-.xml. Total tests 1, failure 1, 
08:32:51 V/ddms: execute: running pm uninstall com.facebook.spectrum.tests.test
08:33:07 V/ddms: execute 'pm uninstall com.facebook.spectrum.tests.test' on 'emulator-5554' : EOF hit. Read: -1
08:33:07 V/ddms: execute: returning

> Task :android:spectrumtests:connectedDebugAndroidTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':android:spectrumtests:connectedDebugAndroidTest'.
> There were failing tests. See the report at: file:///root/react-native/android/spectrumtests/build/reports/androidTests/connected/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

[diegosanchezr]

Some more progress in https://github.com/facebookincubator/spectrum/pull/211

Now at least connectedAndroidTest succeeds sometimes. The remaining issue is related to emulator perf/reliability.

[diegosanchezr]

I'm gonna close this as tests are now passing (emulator a bit flaky sometimes though).