Open xgouchet opened 4 years ago
Hi @szpak ,
I am using the Android fork, which is a wrapper that simply finds the source and test classes differently. That fork works correctly with JUnit 4 tests, but applying this plugin doesn't work.
Ok, it was misled by the link to the original project webpage.
I don't know if the fork support JUnit 5 tests with Android. With enabling verbose mode in PIT (and --info
in Gradle) you should see a message that JUnit 5 plugin is being enabled. Do you have one? If not you might paste PIT related output here.
@koral-- do you have a functional test with JUnit 5 that could confirm that at least in the basic case works?
Btw, @xgouchet here you have a sample JUnit 5 and Kotlin project which works. It's not an Android project, but maybe you will spot any other difference.
So there is an issue in the Android version, which doesn't handle the junit5PluginVersion
attribute, but even when specifying manually the testPlugin
and adding the Junit5 artifact in the buildScript dependencies, it doesn't seem to work.
What line should I be seeing in the logs that confirms that the JUnit5 plugin is indeed loaded and used ?
In the log I see this line :
Running report with ReportOptions
[
targetClasses=[com.example.*],
excludedMethods=[],
excludedClasses=[],
excludedTestClasses=[],
codePaths=[
~/…/build/intermediates/javac/debug/classes,
~/…/build/tmp/kotlin-classes/debug
],
reportDir=~/…/build/reports/pitest,
historyInputLocation=null,
historyOutputLocation=null,
sourceDirs=[
~/…/src/main/java,
~/…/src/main/kotlin
],
classPathElements=[
~/.gradle/caches/modules-2/files-2.1/org.pitest/pitest-junit5-plugin/0.12/6e1ee587a3457b0e7d636a5ce2e554fac62ec99/pitest-junit5-plugin-0.12.jar,
~/.gradle/caches/modules-2/files-2.1/org.pitest/pitest/1.5.0/f8e6d4738f0fca684aaec7c8e0e78e84670fa1bc/pitest-1.5.0.jar,
~/…/build/pitest-android-29-default-values.jar,
~/…/build/intermediates/sourceFolderJavaResources/debug,
~/…/build/intermediates/sourceFolderJavaResources/test/debug,
~/…/build/intermediates/java_res/debug/out,
~/…/build/intermediates/java_res/debugUnitTest/out,
~/…/build/intermediates/unitTestConfig/test/debug,
~/…/build/intermediates/compile_and_runtime_not_namespaced_r_class_jar/debugUnitTest/R.jar,
~/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-launcher/1.5.2/bca9d635ae55b37540e6e26e3e08ea0977516fe3/junit-platform-launcher-1.5.2.jar,
~/.gradle/caches/modules-2/files-2.1/org.junit.vintage/junit-vintage-engine/5.5.2/c3422d2aed908952f9db3e7f3a396776e5a8e017/junit-vintage-engine-5.5.2.jar,
~/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-engine/1.5.2/d5697f6ebe0b4d08c0210b5b98b4e1a40f40dfc6/junit-platform-engine-1.5.2.jar,
~/.gradle/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter/5.5.2/f8561a498ec26b24a3a64aebb6b2307fed000a33/junit-jupiter-5.5.2.jar,
~/.gradle/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter-params/5.5.2/e0659722923bd9fdfa08602e2da22bd5a9d354e8/junit-jupiter-params-5.5.2.jar,
~/.gradle/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter-api/5.5.2/6393db7e4c0265152d8fc4ff146633d1a7d36c47/junit-jupiter-api-5.5.2.jar,
~/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-commons/1.5.2/9703df63b65d415b2a027d43ce908c625b3aedce/junit-platform-commons-1.5.2.jar,
~/…/build/intermediates/javac/debug/classes,
~/…/build/tmp/kotlin-classes/debug,
~/…/build/tmp/kapt3/classes/debug,
~/…/build/tmp/kotlin-classes/debugUnitTest,
~/…/build/tmp/kapt3/classes/debugUnitTest,
~/…/build/intermediates/unmocked-androiddd-sdk-android.jar,
~/…/build/intermediates/javac/debugUnitTest/classes,
~/…/build/intermediates/compile_only_not_namespaced_r_class_jar/debug/R.jar
],
mutators=[STRONGER],
features=[],
dependencyAnalysisMaxDistance=-1,
jvmArgs=[-Djava.awt.headless=true],
numberOfThreads=8,
timeoutFactor=1.25,
timeoutConstant=4000,
targetTests=[^com\.example\..*Test$],
loggingClasses=[kotlin.jvm.internal],
maxMutationsPerClass=0,
verbose=true,
failWhenNoMutations=true,
outputs=[XML, HTML],
groupConfig=TestGroupConfig [excludedGroups=[],
includedGroups=[]],
fullMutationMatrix=false,
mutationUnitSize=0,
shouldCreateTimestampedReports=false,
detectInlinedCode=false,
exportLineCoverage=false,
mutationThreshold=0,
coverageThreshold=0,
mutationEngine=gregor,
javaExecutable=null,
includeLaunchClasspath=false,
properties={},
maxSurvivors=-1,
excludedRunners=[],
includedTestMethods=[],
testPlugin=junit5,
useClasspathJar=false,
skipFailingTests=false
]
Is there something missing ?
It in general looks ok. There could be some Android-related quirks. Maybe @koral-- will have any idea.
I confirm that it junit5 plugin does not work with Android projects. I'll try to figure out what is the exact reason.
Quite a long time but I've managed to find a time to investigate that.
It seems that the workaround consists of 2 steps:
pitest.testPlugin = "junit5"
and pitest "org.pitest:pitest-junit5-plugin:0.12"
buildscript dependency)pitestTestCompile
buildscript dependencySee the commit autolinked above and the whole functional test where it is located.
Nice @koral--!
Based on your research, why pitestTestCompile "org.junit.jupiter:junit-jupiter-engine:$junit5Version"
is required? Is it a problem with junit5 detection by the plugin (only in Android projects?) or that dependency should be added by default for some other reasons?
Normally (not taking pitest into account) that dependency has to be added to testRuntimeOnly
configuration and there is a 3rd party gradle plugin to make it work in Android projects: https://github.com/mannodermaus/android-junit5#setup.
It seems that the more elegant solution in the plugin (apart from making junit5PluginVersion
work) is to add everything from test runtime classpath to additionalClasspath
of pitest task.
I was able to fix that issue.
Now it is possible to just specify junit5PluginVersion
without any additional properties specific to pitest.
I'm trying to run PITest with the JUnit 5 plugin in an Android/Kotlin project. I'm applying the JUnit 5 project using the recommended approach
But when I run PITest, the following outputs is given. It does find my 887 test classes, and can generate mutation, but somehow it doesn't recognizes the tests from my Kotlin classes. Is there a way to make the plugin more verbose as to whay it doesn't find my
@Test
annotated methods ?