search

trevjonez / composer-gradle-plugin

Gradle task type and plugin for interacting with https://github.com/gojuno/composer
Apache License 2.0
54 stars 18 forks source link

Tests duplication and wrong count #47

Closed aleien closed 5 years ago

aleien commented 5 years ago

Hello!

First of all, thank you very mush for developing this plugin!

We are actively writing tests for our app and decided we could use some parallel execution. We set up composer, but we're experiencing some issues with tests count. We have 365 tests (not counting parametrized as separate tests), but running them with composer gives us difference results each time.

Examples: Original image

Composer run 1 image

Composer run 2 (same branch) image

Seems like there are appearing some duplicating tests with 0 time length image image

Our setup: Composer plugin v.10.0 Gradle 5.1.1 Android Build Tools 3.3.0

junit4 parametrized tests orchestrator (support version)

Composer setup

    instrumentationArgument("clearPackageData", "true")
    configs {
        devDebug {
            withOrchestrator true
            outputDirectory project.properties['test.reports.dir'] + "/composer"
        }
    }
}

Although, it seems more like composer problem ): Could you, please, tell what version of composer are you using for plugin v10.0?

trevjonez commented 5 years ago

version 0.10.0 was using 0.5.0 you might try again with version 0.12.0 which uses 0.6.0 (will force you to AGP 3.4.1) or add the higher version to the composer configurations

dependencies {
 composer "com.gojuno.composer:composer:0.6.0"
}

also make sure you tell composer about the orchestrator via

composer {
  withOrchestrator true
}
aleien commented 5 years ago

Thanks for fast reply!

Won't that force us to use androidX orchestrator?

trevjonez commented 5 years ago

I believe you are correct it would force an update to androidX. I do recommend you update as soon as possible though. Going to close this for now is there is nothing actionable for this repo as of now. I may be worth checking or creating an issue directly on composer. https://github.com/gojuno/composer/issues