Composer tasks can't be run in parallel

[plastiv]

I'm not sure how common use case is but running multiple composer gradle tasks in parallel results in issue because they are state (android device or emulator) dependant.

Background on a goal I want to achieve. I want to run all android tests across all gradle modules. We started modularization journey sometime ago but we were keeping the tests in main apk module until now. When moving android tests into corresponding modules we start hitting the wall with infrastructure support. Like having common results report for example.

Current issue:

• For a multi module project (Gradle 6.0.1, AGP 3.5.3, CGP 0.13.1)

rootprojectdir
├── main
│   ├── build.gradle
├── storage
│   ├── build.gradle

• Running android tests with orchestrator

apply plugin: 'com.trevjonez.composer'

composer {
    withOrchestrator true
    instrumentationArgument('clearPackageData', 'true') // orchestrator param
}

dependencies {
    androidTestUtil "androidx.test:orchestrator:1.2.0"
    composer "com.gojuno.composer:composer:0.6.0"
}

• in parallel mode ./gradlew testDebugComposer --parallel

Results in error.

My current understanding of the issue is:

• Gradle treats :main:testDebugComposer and :storage:testDebugComposer as independent
• second task executes adb install -r orchestrator-1.2.0.apk again while first task is still running
• adb install force stops orchestrator process on the device preventing it from finishing test run gracefully

What to do with it is an open question.

• maybe adding to readme that composer should be run with ./gradlew testDebugComposer --no-parallel explicitly is enough
• maybe there should be a parent task configuration which would be multimodule aware
• maybe installing orchestrator should be lazy and composer task should check if apk is already there

[trevjonez]

seems we need some sort of global resource locking to ensure only one composer process can interact with a given device at any time. this shouldn't be too hard to add. as a workaround you would simply need to invoke gradle multiple times serially. you can still assemble the APK's in parallel but just be sure to keep the composer instances singular.

--no-parallel is a non starter because it impacts all tasks not just the composer tasks.

I am thinking this should be a composer handled item rather than plugin. probably as simple as a lock file in ~/.android for each device. once the file is gone the device is available and the next process can grab it and go. now, how does one double check lock at the fs/os level 🤔

[plastiv]

I just noticed that something similar is being discussed by Gradle https://github.com/gradle/gradle/issues/7047 with the proposed API https://github.com/gradle/gradle/pull/9914 . Requires Gradle 6.1 RC1 and task workers as of now.

[trevjonez]

That would work at the Gradle invocation level to get at most one composer task at a time. Likely a sufficient solution if we assume from the plugin that composer can run things across devices efficiently. (Not an assumption that is alway true, but not entirely false either)

[dannybduval]

From what I've seen on this, Orchestrator itself crashes. Each Orchestrated run installs the same apk from Cache, e.g.:

Task :storage:test${application}DebugComposer
[emulator-5554] Installing apk... pathToApk = ${GRADLE_HOME}/caches/modules-2/files-2.1/androidx.test/orchestrator/1.2.0/e3edf4e08d2d3db127c1b56bbe405b90b64daa5/orchestrator-1.2.0.apk
[emulator-5554] Successfully installed apk in 5 seconds, pathToApk = $GRADLE_HOME/caches/modules-2/files-2.1/androidx.test/orchestrator/1.2.0/e3edf4e08d2d3db127c1b56bbe405b90b64daa5/orchestrator-1.2.0.apk
[emulator-5554] Installing apk... pathToApk = ${GRADLE_HOME}/caches/modules-2/files-2.1/androidx.test.services/test-services/1.2.0/58c9ac48a725a1a0437f4f6be352a42f60ed5a7d/test-services-1.2.0.apk
[emulator-5556] Installing apk... pathToApk = ${GRADLE_HOME}/caches/modules-2/files-2.1/androidx.test/orchestrator/1.2.0/e3edf4e08d2d3db127c1b56bbe405b90b64daa5/orchestrator-1.2.0.apk
[emulator-5554] Successfully installed apk in 1 second, pathToApk = ${GRADLE_HOME}/caches/modules-2/files-2.1/androidx.test.services/test-services/1.2.0/58c9ac48a725a1a0437f4f6be352a42f60ed5a7d/test-services-1.2.0.apk

> Task :main:test${applicatin}DebugComposer
[emulator-5554] Successfully installed apk in 1 second, pathToApk = ${GRADLE_HOME}/caches/modules-2/files-2.1/androidx.test/orchestrator/1.2.0/e3edf4e08d2d3db127c1b56bbe405b90b64daa5/orchestrator-1.2.0.apk
[emulator-5554] Installing apk... pathToApk = ${GRADLE_HOME}/caches/modules-2/files-2.1/androidx.test.services/test-services/1.2.0/58c9ac48a725a1a0437f4f6be352a42f60ed5a7d/test-services-1.2.0.apk
[emulator-5554] Successfully installed apk in 2 seconds, pathToApk = ${GRADLE_HOME}/caches/modules-2/files-2.1/androidx.test.services/test-services/1.2.0/58c9ac48a725a1a0437f4f6be352a42f60ed5a7d/test-services-1.2.0.apk
[emulator-5556] Successfully installed apk in 4 seconds, pathToApk = ${GRADLE_HOME}/caches/modules-2/files-2.1/androidx.test/orchestrator/1.2.0/e3edf4e08d2d3db127c1b56bbe405b90b64daa5/orchestrator-1.2.0.apk
[emulator-5556] Installing apk... pathToApk = ${GRADLE_HOME}/caches/modules-2/files-2.1/androidx.test.services/test-services/1.2.0/58c9ac48a725a1a0437f4f6be352a42f60ed5a7d/test-services-1.2.0.apk

Depending on which module installs last, it will run, all previous will fail (should follow dependency chain).

If you go to the logcat of the device that crashed, you'll see that the underlying issue seems to be more along the lines of attempting to install the same APK twice in this process.

The likely path I see here is this (based on device log cat):

1. storage runs, installs APKs for orchestrator
2. main gets to the point of running tests, install of APKs for orchestrator kill the process current orchestrator APKs. This results in the run for storage to fail

[trevjonez]

i'll see if I can work something up on my commute home today.

[trevjonez]

give rc2 a try and let me know how it goes.

[dannybduval]

https://jitpack.io/com/github/trevjonez/composer-gradle-plugin/1.0.0-rc2/build.log

* What went wrong:
Execution failed for task ':composer:composer:npmInstall'.
> A problem occurred starting process 'command 'npm''

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 33s
8 actionable tasks: 4 executed, 4 up-to-date

Publishing build scan...
https://gradle.com/s/qy55xnhppt65i

Build tool exit code: 0
Looking for artifacts...
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Dhttps.protocols=TLSv1.2
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Dhttps.protocols=TLSv1.2
Looking for pom.xml in build directory and ~/.m2
Found artifact: com.github.trevjonez:commander-android:1.0.0-rc2
2020-01-04T02:24:16.884505536Z
Exit code: 0

ERROR: No build artifacts found
Expected artifacts in: $HOME/.m2/repository/com/github/trevjonez/commander-android/1.0.0-rc2

[trevjonez]

jitpack support might be over now that I have to build the juno libraries. can you pull from jcenter?

[trevjonez]

looks like I have some deadlocking still. so no need to try any further right now. I've updated the readme as well to not direct people to jitpack anymore.

[dannybduval]

I can point the repository to whichever is needed. I've already got dependencies coming from both places.

[trevjonez]

ok 1.0.0-rc03 should do it. you will need to update the plugin inclusion to use the one from the plugin portal rather than jitpack, then ensure jcenter is available on your buildscript classpath.

[dannybduval]

I can get to that, but I think that the core issue is still that Orchestrator itself will not run test in parallel on the same device.

We've got 6 modules and if I try and run in parallel, I get varying instances of this sort of failure:

E TestRunner: failed: Test mechanism
E TestRunner: ----- begin exception -----
E TestRunner: java.lang.NullPointerException: Unable to process test notification. No ListenerManager
E TestRunner:   at android.os.Parcel.readException(Parcel.java:1605)
E TestRunner:   at android.os.Parcel.readException(Parcel.java:1552)
E TestRunner:   at androidx.test.runner.internal.deps.aidl.BaseProxy.transactAndReadExceptionReturnVoid(BaseProxy.java:66)
E TestRunner:   at androidx.test.orchestrator.callback.OrchestratorCallback$Stub$Proxy.sendTestNotification(OrchestratorCallback.java:95)
E TestRunner:   at androidx.test.orchestrator.instrumentationlistener.OrchestratedInstrumentationListener.sendTestNotification(OrchestratedInstrumentationListener.java:167)
E TestRunner:   at androidx.test.orchestrator.instrumentationlistener.OrchestratedInstrumentationListener.testRunFinished(OrchestratedInstrumentationListener.java:104)
E TestRunner:   at org.junit.runner.notification.SynchronizedRunListener.testRunFinished(SynchronizedRunListener.java:42)
E TestRunner:   at org.junit.runner.notification.RunNotifier$2.notifyListener(RunNotifier.java:103)
E TestRunner:   at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72)
E TestRunner:   at org.junit.runner.notification.RunNotifier.fireTestRunFinished(RunNotifier.java:100)
E TestRunner:   at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
E TestRunner:   at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
E TestRunner:   at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
E TestRunner:   at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:392)
E TestRunner:   at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879)

Unless the underlying framework is aware enough to create separate instances of each binder, you'll run over each (at least how I'm seeing it when I try and go through the decompiled code (I could most definitely be wrong)

[trevjonez]

is this with rc03? it should be locking the entire duration of install -> run -> data pulling