Pitest with simple Groovy project

[mariogarcia]

Hi:

First of all, great work, this project is amazing!

I'm trying to use pitest in a Groovy (Gradle based) project but although it always seems to find Spock specifications it never creates any mutation. I've created a basic Groovy project (which I've attached to this issue pitestgroovyproject.zip) to reproduce the problem.

This is the pitest configuration

pitest {
    targetTests   = ['myproject.*']
    targetClasses = ['myproject.*']
    threads       = 4
    outputFormats = ['html']
}

The console output:

clean
:compileJava UP-TO-DATE
:compileGroovy
:processResources UP-TO-DATE
:classes
:jar
:assemble
:compileTestJava UP-TO-DATE
:compileTestGroovy
:processTestResources UP-TO-DATE
:testClasses
:test
:check
:build
:pitest
12:10:49 PM PIT >> FINE : Running report with ReportOptions [targetClasses=[^myproject\..*$], excludedMethods=[], excludedClasses=[], codePaths=[/home/dev/ws/build/classes/main], reportDir=/home/dev/ws/build/reports/pitest, historyInputLocation=null, historyOutputLocation=n
ull, sourceDirs=[/home/dev/ws/src/main/resources, /home/dev/ws/src/main/java, /home/dev/ws/src/main/groovy], classPathElements=[/home/dev/.gradle/caches/modules-2/files-2.1/org.pitest/pitest/1.1.9/b9dcaaf81257e9f436c73e59579461d8405e34e5/pitest-1.1.9.jar, /home/dev/ws/build
/classes/test, /home/dev/ws/build/resources/test, /home/dev/ws/build/classes/main, /home/dev/ws/build/resources/main, /home/dev/.gradle/caches/modules-2/files-2.1/org.codehaus.groovy/groovy-all/2.4.6/478feadca929a946b2f1fb962bb2179264759821/groovy-all-2.4.6.jar, /home/dev/.
gradle/caches/modules-2/files-2.1/org.spockframework/spock-core/1.0-groovy-2.4/ceaa8b69f274ed3de24da3e6a6c86f673b426d1a/spock-core-1.0-groovy-2.4.jar, /home/dev/.gradle/caches/modules-2/files-2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-4.12.jar, /hom
e/dev/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-1.3.jar], mutators=[], dependencyAnalysisMaxDistance=-1, mutateStaticInitializers=false, jvmArgs=[], numberOfThreads=4, timeoutFactor=1.25, timeout
Constant=4000, targetTests=[^myproject\..*$], loggingClasses=[], maxMutationsPerClass=0, verbose=true, failWhenNoMutations=true, outputs=[html], groupConfig=TestGroupConfig [excludedGroups=[], includedGroups=[]], mutationUnitSize=0, shouldCreateTimestampedReports=true, dete
ctInlinedCode=false, exportLineCoverage=false, mutationThreshold=0, coverageThreshold=0, mutationEngine=gregor, javaExecutable=null, includeLaunchClasspath=false]
12:10:49 PM PIT >> FINE : System class path is /home/dev/.gradle/caches/modules-2/files-2.1/org.pitest/pitest-command-line/1.1.9/ed8bd8e7802ac2d037cc795f205207915fd8c777/pitest-command-line-1.1.9.jar:/home/dev/.gradle/caches/modules-2/files-2.1/org.pitest/pitest/1.1.9/b9dca
af81257e9f436c73e59579461d8405e34e5/pitest-1.1.9.jar:/home/dev/.gradle/caches/modules-2/files-2.1/junit/junit/4.11/4e031bb61df09069aeb2bffb4019e7a5034a4ee0/junit-4.11.jar:/home/dev/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f7
1acb49bf010a0/hamcrest-core-1.3.jar
12:10:49 PM PIT >> FINE : Maximum available memory is 2604 mb
12:10:49 PM PIT >> FINE : MINION : Installing PIT agent

12:10:49 PM PIT >> INFO : Sending 1 test classes to minion
12:10:49 PM PIT >> INFO : Sent tests to minion
12:10:49 PM PIT >> INFO : MINION : 12:10:49 PM PIT >> INFO : Checking environment
12:10:49 PM PIT >> FINE : Expecting 1 tests classes from parent
12:10:49 PM PIT >> FINE : Tests classes received

12:10:49 PM PIT >> INFO : MINION : 12:10:49 PM PIT >> INFO : Found  1 tests

12:10:49 PM PIT >> INFO : MINION : 12:10:49 PM PIT >> INFO : Dependency analysis reduced number of potential tests by 0

12:10:49 PM PIT >> INFO : MINION : 12:10:49 PM PIT >> INFO : 1 tests received

12:10:49 PM PIT >> INFO : MINION : 12:10:49 PM PIT >> FINE : Running 1 units

12:10:49 PM PIT >> INFO : MINION : 12:10:49 PM PIT >> FINE : Gathering coverage for test Description [testClass=myproject.LibraryTest, name=myproject.LibraryTest]

12:10:49 PM PIT >> FINE : Coverage generator Minion exited ok
12:10:49 PM PIT >> INFO : Calculated coverage in 0 seconds.
12:10:49 PM PIT >> FINE : Used memory after coverage calculation 19 mb
12:10:49 PM PIT >> FINE : Free Memory after coverage calculation 157 mb
12:10:49 PM PIT >> INFO : Created  0 mutation test units
Exception in thread "main" org.pitest.help.PitHelpError: No mutations found. This probably means there is an issue with either the supplied classpath or filters.
See http://pitest.org for more details.
        at org.pitest.mutationtest.tooling.MutationCoverage.checkMutationsFound(MutationCoverage.java:272)
        at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:136)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:103)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:45)
        at org.pitest.mutationtest.commandline.MutationCoverageReport.runReport(MutationCoverageReport.java:76)
        at org.pitest.mutationtest.commandline.MutationCoverageReport.main(MutationCoverageReport.java:45)
:pitest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':pitest'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

Thanks Mario

[hcoles]

Hi Mario,

At the moment pitest intentionally avoids mutating groovy code as it knows it can't do a very good job of it. It would create a lot of junk mutations in code that didn't map back directly to anything expressed in code by a developer.

It might be possible to filter those mutations out or create groovy specific mutation operators that give better results. The last time I looked at this the invokedynamic operator had just arrived on the JVM and the implementation of groovy was in transition.

I assume things have settled down now so it might be time to take another look.

[mariogarcia]

Thanks for the explanation Henry! I understand.

I'll keep an eye on the repo to see if there is any progress on this. Anyway in case you start the task, it would be nice to have some kind of feedback to see if there's anything the Groovy community could do in order to help you.

Again thanks for your patience Mario

[dbaravalle]

+1

[sarah-jay-peper]

I have a similar problem, as we have groovy tests for java code. Again it finds the test specifications but does not mutate the java code. Probably because pitest is assuming the code is groovy as well. Is there a way to override this behaviour?

[szpak]

@jasontrublu Mutating Java code by Groovy/Spock tests should work (at least in a Gradle-based project). The original issue is about mutating Groovy production code. Please raise a separate issue in gradle-pitest-plugin providing link to your project or SSCCE.

[sarah-jay-peper]

that's interesting. When I tried to build a simple example project, it actually worked. Now I need to find out what's different in our bigger project (which is closed source)

[dylanwatsonsoftware]

I found that if I ran mvn compile org.pitest:pitest-maven:mutationCoverage to ensure that the groovy files were converted to Java first, then it worked ok.

[prosidney]

Any Ideia of when pitest would support mutation of groovy code?

[hcoles]

@prosidney It would require someone to fund my time to implement it, or for someone else to pick it up.