search

gojuno / composer

Reactive Android Instrumentation Test Runner. Archived. Marathon is recommended as an alternative (https://github.com/Malinskiy/marathon).
Apache License 2.0
546 stars 45 forks source link

Unable to run tests #102

Closed Kregap closed 6 years ago

Kregap commented 6 years ago

Hi, I'm would really love to try out your tool, as I'm having some significant stability issues with spoon usage and due to dependencies am unable to use Test Orchestrator. Unfortunately I'm unable to start any test and have no idea what is wrong.

I tried it on project where I'm using spoon ATM and on dummy project to not use our custom runner and have most basic setup possible.

This is how I'm running it:

$ ./gradlew clean assembleDebug assembleAndroidTest
$ java -jar ~/Downloads/composer-0.2.6.jar \
  --apk "app/build/outputs/apk/debug/app-debug.apk" \
  --test-apk "app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk" \
  --test-package com.example.dummyproject \
  --test-runner android.support.test.runner.AndroidJUnitRunner

I've tried with providing runner with package specified too:

--test-runner com.example.dummyproject.test/android.support.test.runner.AndroidJUnitRunner

In both cases there seems to be problem with Unable to find instrumentation info (...), so I assume I'm providing runner in wrong way.

[Tue Oct 10 10:25:15 CEST 2017]: 1 connected adb device(s): [AdbDevice(id=emulator-5554, online=true)]
[Tue Oct 10 10:25:15 CEST 2017]: [emulator-5554] Installing apk... pathToApk = app/build/outputs/apk/debug/app-debug.apk
[Tue Oct 10 10:25:18 CEST 2017]: [emulator-5554] Successfully installed apk in 2 seconds, pathToApk = app/build/outputs/apk/debug/app-debug.apk
[Tue Oct 10 10:25:18 CEST 2017]: [emulator-5554] Installing apk... pathToApk = app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
[Tue Oct 10 10:25:21 CEST 2017]: [emulator-5554] Successfully installed apk in 2 seconds, pathToApk = app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
[Tue Oct 10 10:25:21 CEST 2017]: [emulator-5554] Starting tests...
[Tue Oct 10 10:25:22 CEST 2017]: [emulator-5554] Error during tests run: java.lang.IllegalStateException: Process [/Users/kregap/Android_Tools/Android_SDK/platform-tools/adb, -s, emulator-5554, shell, am instrument -w -r com.example.dummyproject/android.support.test.runner.AndroidJUnitRunner] exited with non-zero code 1 android.util.AndroidException: INSTRUMENTATION_FAILED: com.example.dummyproject/android.support.test.runner.AndroidJUnitRunner
        at com.android.commands.am.Am.runInstrument(Am.java:890)
        at com.android.commands.am.Am.onRun(Am.java:400)
        at com.android.internal.os.BaseCommand.run(BaseCommand.java:51)
        at com.android.commands.am.Am.main(Am.java:121)
        at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
        at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:262)
INSTRUMENTATION_STATUS: id=ActivityManagerService
INSTRUMENTATION_STATUS: Error=Unable to find instrumentation info for: ComponentInfo{com.example.dummyproject/android.support.test.runner.AndroidJUnitRunner}
INSTRUMENTATION_STATUS_CODE: -1

Exception in thread "main" java.lang.IllegalStateException: Process [/Users/kregap/Android_Tools/Android_SDK/platform-tools/adb, -s, emulator-5554, shell, am instrument -w -r com.example.dummyproject/android.support.test.runner.AndroidJUnitRunner] exited with non-zero code 1 android.util.AndroidException: INSTRUMENTATION_FAILED: com.example.dummyproject/android.support.test.runner.AndroidJUnitRunner
        at com.android.commands.am.Am.runInstrument(Am.java:890)
        at com.android.commands.am.Am.onRun(Am.java:400)
        at com.android.internal.os.BaseCommand.run(BaseCommand.java:51)
        at com.android.commands.am.Am.main(Am.java:121)
        at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
        at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:262)
INSTRUMENTATION_STATUS: id=ActivityManagerService
INSTRUMENTATION_STATUS: Error=Unable to find instrumentation info for: ComponentInfo{com.example.dummyproject/android.support.test.runner.AndroidJUnitRunner}
INSTRUMENTATION_STATUS_CODE: -1

        at com.gojuno.commander.os.ProcessesKt$process$1.call(Processes.kt:97)
        at com.gojuno.commander.os.ProcessesKt$process$1.call(Processes.kt)
        at rx.internal.operators.OnSubscribeCreate.call(OnSubscribeCreate.java:72)
        at rx.internal.operators.OnSubscribeCreate.call(OnSubscribeCreate.java:32)
        at rx.Observable.unsafeSubscribe(Observable.java:10256)
        at rx.internal.operators.OperatorSubscribeOn$SubscribeOnSubscriber.call(OperatorSubscribeOn.java:100)
        at rx.internal.schedulers.CachedThreadScheduler$EventLoopWorker$1.call(CachedThreadScheduler.java:230)
        at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
plastiv commented 6 years ago

This project is not compatible with Test Orchestrator either: https://github.com/gojuno/composer/issues/90

Kregap commented 6 years ago

Mentioning Test Orchestrator was just to describe my reasoning, sorry for confusion. I actually gave up on using it for now and wanted just to switch from Spoon to Composer. This example project is setup to just use Espresso with AndroidJUnitRunner for AndroidJUnit4.

artem-zinnatullin commented 6 years ago

Questions:

Kregap commented 6 years ago

Thanks for addressing my issue so fast.

I've created example project to try out in simplest setup for beginning. It's available in: composer-usage-example.

  1. I'm able to run tests with ./gradlew connectedAndroidTest without problem. Additionally I've added 2 scripts so you could check how I'm trying to run tests, you should be able to run them from main project directory: a) running directly with instrumentation - scritps/instrumentation-runner.sh, running without problem, b) running with Composer - scripts/composer-runner.sh, failing with same Unable to find instrumentation info (...) as in original issue description.
  2. It was declared - line with declaration.
  3. Checked it, seems correct - package declaration.
  4. I've used Composer v0.2.6 from jcenter linked in Composers README.md.

About following log:

[Tue Oct 10 22:47:10 CEST 2017]: [emulator-5554] Error during tests run: java.lang.IllegalStateException: Process [/opt/android-sdk/platform-tools/adb, -s, emulator-5554, shell, am instrument -w -r com.example.kregap.dummyproject/android.support.test.runner.AndroidJUnitRunner] exited with non-zero code 1 android.util.AndroidException: INSTRUMENTATION_FAILED: com.example.kregap.dummyproject/android.support.test.runner.AndroidJUnitRunner

shouldn't Composer call instrumentation in test package (com.example.kregap.dummyproject.test/android.support.test.runner.AndroidJUnitRunner) instead of one in app package (com.example.kregap.dummyproject/android.support.test.runner.AndroidJUnitRunner)?

PS Sorry for closing and reopening issue, I've missclicked "Close issue" button... :|

yunikkk commented 6 years ago

Hi, try to set --test-package com.example.kregap.dummyproject.test in https://github.com/Kregap/composer-usage-example/blob/master/scripts/composer-runner.sh.

Kregap commented 6 years ago

Oh... Ok, so that's classical issue between keyboard and monitor. :) Initially i though that --test-package argument is for specifying package in app where tests are implemented for grouping purposes, but that's for actual test app package to build instrumentation command properly.

Thanks a lot for resolving my issue so fast.

yunikkk commented 6 years ago

We could probably state that explicitly in Readme... Or, better, finally add parsing from test apk)

artem-zinnatullin commented 6 years ago

Definitely would be great to start parsing apk for that :)

Maybe with help of apkanalyzer, but that would be additional 3rd-party dependency that could have breaking changes and so on

trevjonez commented 6 years ago

or use the gradle plugin which can pull it directly from the android plugin declarations.

artem-zinnatullin commented 6 years ago

Hey, that's cheating! :D

Kregap commented 6 years ago

I'm not familiar with Composer code and don't know what tools are available for you, but if you can use Android Asset Packaging Tool (aapt) available in Android SDK ($ANDROID_HOME/build-tools/x.y.z/aapt) and adb shell then you could obtain both test package name and instrumentation name in following way:

$ $ANDROID_HOME/build-tools/x.y.z/aapt dump badging app-debug-androidTest.apk | grep package
package: name='com.example.kregap.dummyproject.test' versionCode='' versionName='' platformBuildVersionName='8.0.0'

$ adb shell cmd package list instrumentation | grep instrumentation:com.example.kregap.dummyproject.test
instrumentation:com.example.kregap.dummyproject.test/android.support.test.runner.AndroidJUnitRunner (target=com.example.kregap.dummyproject)

That is as I said assuming you can use those tools in Composer.

yunikkk commented 6 years ago

@Kregap that seems to be really easy to implement, thanks for the hint!

nacuteodor commented 6 years ago

@Kregap, @artem-zinnatullin I ran the the composer like this: testRunner=composer-0.3.1.jar java -jar ${testRunner} \ --apk ${APK} \ --test-apk ${TEST_APK} \ --test-package "com.android.emulator.smoketests" \ --test-runner android.support.test.runner.AndroidJUnitRunner \ --output-directory test-outputs \ --instrumentation-arguments class ${currentSuiteTest} \ --shard false \ --install-timeout 600 \ --verbose-output true \ --keep-output-on-exit

12:42:56 [Mon Jul 09 12:42:56 UTC 2018]: Args(appApkPath=..., testApkPath=..., testPackage=com.android.emulator.smoketests, testRunner=android.support.test.runner.AndroidJUnitRunner, shard=false, outputDirectory=test-outputs, instrumentationArguments=[class, ${currentSuiteTest}], verboseOutput=true, keepOutputOnExit=true, devices=[], devicePattern=, installTimeoutSeconds=300) 12:42:56 [Mon Jul 09 12:42:56 UTC 2018]: 1 connected adb device(s): [AdbDevice(id=emulator-5556, model=Android SDK built for x86, online=true)] 12:42:56 [Mon Jul 09 12:42:56 UTC 2018]: [emulator-5556] Installing apk... pathToApk = ... 12:43:07 [Mon Jul 09 12:43:07 UTC 2018]: [emulator-5556] Successfully installed apk in 10 seconds, pathToApk = ... 12:43:07 [Mon Jul 09 12:43:07 UTC 2018]: [emulator-5556] Installing apk... pathToApk = ... 12:43:10 [Mon Jul 09 12:43:10 UTC 2018]: [emulator-5556] Successfully installed apk in 3 seconds, pathToApk = ... 12:43:10 [Mon Jul 09 12:43:10 UTC 2018]: [emulator-5556] Starting tests... 12:43:11 [Mon Jul 09 12:43:11 UTC 2018]: [emulator-5556] Test run finished, 0 passed, 0 failed, took 1 second. 12:43:11 [Mon Jul 09 12:43:11 UTC 2018]: Generating HTML report... 12:43:11 [Mon Jul 09 12:43:11 UTC 2018]: HTML report generated, took 0 seconds. 12:43:11 [Mon Jul 09 12:43:11 UTC 2018]: Test run finished, total passed = 0, total failed = 0, total ignored = 0, took 14 seconds. 12:43:11 [Mon Jul 09 12:43:11 UTC 2018]: Error: 0 tests were run.

Note that ${currentSuiteTest} tests class is not from "com.android.emulator.smoketests" But this command runs 0 tests. How did you specify the tests class you want to run? Could you help me, please? Thanks

nacuteodor commented 6 years ago

I was able to pass the right test-package and test-runner after listing the test runners: adb shell pm list instrumentation