search

gojuno / composer

Reactive Android Instrumentation Test Runner. Archived. Marathon is recommended as an alternative (https://github.com/Malinskiy/marathon).
Apache License 2.0
544 stars 45 forks source link

Dynamic test sharding. #66

Open artem-zinnatullin opened 7 years ago

artem-zinnatullin commented 7 years ago

Currently Composer supports only static sharding which is a feature of Android Test Support library.

Static sharding tend to be inefficient since it is impossible to know statically duration of each individual test execution, which means that some devices may run out of tests to execute faster than others which creates bottleneck for the overall test run time.

Composer can communicate with test runner on a device and run tests in interactive mode when Composer gives commands to test runner on a device to run particular test which should decrease overall test run time and increase resource usage efficiency.

artem-zinnatullin commented 7 years ago

Btw, we've (@yunikkk) tried to launch each test through separate adb am instrumentation call but that caused overhead of 3-4 seconds for each test which is simply unacceptable for us.

Looks like new Test Orchestrator made by Google will have the same issue.

So idea of talking to JUnitRunner on a device through BroadCasts is still valid and should be the most efficient one.