search

szpak / gradle-pitest-plugin

Gradle plugin for PIT Mutation Testing
http://gradle-pitest-plugin.solidsoft.info/
217 stars 58 forks source link

No mutations found with official java-test-fixtures plugin applied #173

Closed emlun closed 4 years ago

emlun commented 4 years ago

Gradle 5.6 added a java-test-fixtures plugin, which when applied makes pitest fail to generate any mutations. This seems to be a classpath issue.

To reproduce, simply apply plugin: 'java-test-fixtures'. See this repository for a minimal example: https://github.com/emlun/issue-gradle-pitest-test-fixtures

It seems to boil down to this difference in classpath between not having java-test-fixtures applied (top) and having it applied (bottom). Notice how the latter includes the main source set JAR instead of the classes directory:

/home/emlun/dev/gradle-pitest-test-fixtures/build/classes/java/main,/home/emlun/dev/gradle-pitest-test-fixtures/build/resources/main

/home/emlun/dev/gradle-pitest-test-fixtures/build/libs/gradle-pitest-test-fixtures-test-fixtures.jar,/home/emlun/dev/gradle-pitest-test-fixtures/build/libs/gradle-pitest-test-fixtures.jar

As such, this looks like a candidate for forwarding as a bug in Gradle rather than a bug in the gradle-pitest plugin itself.

szpak commented 4 years ago

Thanks for your detailed analysis and a sample project!

You are right that adding the java-test-fixtures plugin adds test-fixtures.jar to a classpath, but also removes main classes (and resources). In that case, I would expect to have main sourceSet on the runtimeClasspath for testFixtures sourceSet (see project.sourceSets.testFixtures.runtimeClasspath.files), but it's not (then the pitest plugin testFixtures could be detected and added to testSourceSets by default).

As a result, I agree that it might be good to report that problem to Gradle - maybe they explain how it should be handled.

Btw, as a temporary workaround you can use:

pitest {
  ...
  testSourceSets.add(project.sourceSets.main)
}

task mainClasses {}

to bring it back online. The last line is required as the plugin is looking to the *classes tasks for defined test sourceSets to depend on. main is specific and It might be first checked if that task exists.

emlun commented 4 years ago

Thanks for the reply! The suggested workaround also works just fine, thanks!

I'll forward this on to Gradle, then. It also turns out this used to work in Gradle 5.6.4, but stopped working in 6.0.1 (I wasn't able to test with 6.0.0). This commit is the only change to the java-test-fixtures plugin between the 5.6.4 and 6.0.1 tags, so that change seems to be the cause.