search

gojuno / composer

Reactive Android Instrumentation Test Runner. Archived. Marathon is recommended as an alternative (https://github.com/Malinskiy/marathon).
Apache License 2.0
546 stars 45 forks source link

If the application process crashes, the JUnit xml file is not generated #154

Open nacuteodor opened 6 years ago

nacuteodor commented 6 years ago

Ran the tool like this:

java -jar composer-0.3.3.jar --apk ${APP_APK} --test-apk ${TEST_APK} --output-directory test-outputs --instrumentation-arguments class ${SUITE_CLASS_NAME} --shard false --install-timeout 300 --verbose-output true
15:17:41 [Fri Jul 06 15:17:41 UTC 2018]: Args(appApkPath=... testApkPath=..., testRunner=, shard=false, outputDirectory=test-outputs, instrumentationArguments=[class, ...], verboseOutput=true, keepOutputOnExit=false, devices=[], devicePattern=, installTimeoutSeconds=300)
15:17:41 [Fri Jul 06 15:17:41 UTC 2018]: 1 connected adb device(s): [AdbDevice(id=emulator-5556, model=Android SDK built for x86, online=true)]
15:17:41 [Fri Jul 06 15:17:41 UTC 2018]: [emulator-5556] Installing apk... pathToApk = ...
15:18:08 [Fri Jul 06 15:18:08 UTC 2018]: [emulator-5556] Successfully installed apk in 27 seconds, pathToApk = ...
15:18:08 [Fri Jul 06 15:18:08 UTC 2018]: [emulator-5556] Installing apk... pathToApk = ...
15:18:14 [Fri Jul 06 15:18:14 UTC 2018]: [emulator-5556] Successfully installed apk in 6 seconds, pathToApk = ...
15:18:14 [Fri Jul 06 15:18:14 UTC 2018]: [emulator-5556] Starting tests...
15:23:10 [Fri Jul 06 15:23:10 UTC 2018]: [emulator-5556] Test 1/7 passed in 4 minutes 51 seconds: ...
15:26:52 [Fri Jul 06 15:26:52 UTC 2018]: [emulator-5556] Test 2/7 passed in 3 minutes 41 seconds: ...
15:30:39 [Fri Jul 06 15:30:39 UTC 2018]: [emulator-5556] Test 3/7 passed in 3 minutes 46 seconds: ...
15:34:32 [Fri Jul 06 15:34:32 UTC 2018]: [emulator-5556] Test 4/7 passed in 3 minutes 52 seconds: ...
15:37:16 [Fri Jul 06 15:37:16 UTC 2018]: [emulator-5556] Test 5/7 passed in 2 minutes 42 seconds: ...
15:39:36 [Fri Jul 06 15:39:36 UTC 2018]: [emulator-5556] Test 6/7 failed in 2 minutes 19 seconds: ...
15:39:36 [Fri Jul 06 15:39:36 UTC 2018]: [emulator-5556] Error during tests run: java.lang.Exception: Application process crashed. Check Logcat output for more details.
15:39:36 Exception in thread "main" java.lang.RuntimeException: java.lang.Exception: Application process crashed. Check Logcat output for more details.
15:39:36    at rx.exceptions.Exceptions.propagate(Exceptions.java:57)
15:39:36    at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:463)
15:39:36    at rx.observables.BlockingObservable.first(BlockingObservable.java:166)
15:39:36    at com.gojuno.composer.MainKt.runAllTests(Main.kt:187)
15:39:36    at com.gojuno.composer.MainKt.main(Main.kt:65)
15:39:36 Caused by: java.lang.Exception: Application process crashed. Check Logcat output for more details.
15:39:36    at com.gojuno.composer.InstrumentationKt.throwIfError(Instrumentation.kt:67)
15:39:36    at com.gojuno.composer.InstrumentationKt.access$throwIfError(Instrumentation.kt:1)
15:39:36    at com.gojuno.composer.InstrumentationKt$readInstrumentationOutput$2.call(Instrumentation.kt:113)
15:39:36    at com.gojuno.composer.InstrumentationKt$readInstrumentationOutput$2.call(Instrumentation.kt)
15:39:36    at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:69)
15:39:36    at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:77)
15:39:36    at rx.internal.operators.OnSubscribeCreate$BufferEmitter.drain(OnSubscribeCreate.java:366)
15:39:36    at rx.internal.operators.OnSubscribeCreate$BufferEmitter.onNext(OnSubscribeCreate.java:299)
15:39:36    at com.gojuno.composer.FilesKt$tail$1$1.handle(Files.kt:15)
15:39:36    at org.apache.commons.io.input.Tailer.readLines(Tailer.java:528)
15:39:36    at org.apache.commons.io.input.Tailer.run(Tailer.java:459)
15:39:36    at java.lang.Thread.run(Thread.java:748)
15:39:36 Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: INSTRUMENTATION_RESULT: shortMsg=Process crashed.
15:39:36    at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:118)
15:39:36    at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:73)
15:39:36    ... 7 m
simonschiller commented 6 years ago

Also, the HTML is not generated and further tests are not run (#131).

mikepenz commented 5 years ago

Seeing the same problem.

abjerstedt commented 4 years ago

This may or may not be related, but i'm seeing failed tests causing the whole process to crash (even through ATO), which then doesn't generate the xml.

nacuteodor commented 4 years ago

I gave up using composer. I use Android Test Orchestrator for robustness and it will crash only the affected test, but not the entire tests class and the xml is always generated.

abjerstedt commented 4 years ago

@nacuteodor android test orchestrator slows my test builds down by ~300-400%. What are you doing to handle the slow speeds?

nacuteodor commented 4 years ago

@abjerstedt I am not using gradle to build with the Android Test Orchestrator. I just deploy the orchestrator apks to the emulator and then run the tests with the orchestrator. So, that doesn't seem to noticeble slow down the tests runtime. The tests builds are built independently of the Android Test Orchestrator.