Android: tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite.

MonkeyMiguelS commented 5 days ago

I'm having a lot of issues trying to make Pitest run against my Android project. I've solved a lot of problems related with Java/Gradle/Kotlin versions but now I'm stuck in a custom Pitest error:

Exception in thread "main" org.pitest.help.PitHelpError: 69 tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite. See https://pitest.org for more details. at org.pitest.coverage.execute.DefaultCoverageGenerator.verifyBuildSuitableForMutationTesting(DefaultCoverageGenerator.java:127) at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:109) at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:53) at org.pitest.mutationtest.tooling.MutationCoverage.runAnalysis(MutationCoverage.java:149) at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:136) at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:136) at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:63) at org.pitest.mutationtest.commandline.MutationCoverageReport.runReport(MutationCoverageReport.java:98) at org.pitest.mutationtest.commandline.MutationCoverageReport.main(MutationCoverageReport.java:45)

I don't know why am I getting this exception when, as you can see, the tests pass successfully and the line coverage is around 95%:

What am I doing wrong? Where's the issue? For further details, I'm running with Java JetBrains Runtime 17.0.11, Kotlin 1.9.23, Gradle 7.4.2 and pl.droidsonroids.pitest version 0.2.18, and my current pitest configuration block is the following:

pitest {
    targetClasses=['app.*']
    targetTests=["app.domain.*"]
    avoidCallsTo=["kotlin.jvm.internal"]
    mutators=["STRONGER"]
    threads=Runtime.getRuntime().availableProcessors()
    outputFormats=["XML", "HTML"]
    mutationThreshold=70
    coverageThreshold=0
}

Thanks in advance for your answer!

hcoles commented 4 days ago

If you can provide a minimal self contained project that replicates the issue I can take a look.

MonkeyMiguelS commented 4 days ago

It's not possible right now. Is there any other way we can try to solve this?

hcoles / pitest

Android: tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite. #1335