Closed vmassol closed 5 years ago
Sometimes a green test fails with PIT a workaround is to remove the test from the analysis.
The class in question seems to be org.xwiki.job.internal.DefaultJobStatusStoreTest
from the SEVERE
message in the output.
Yes I know how to exclude stuff but IMO there's something to fix to make the tool better (hence the reporting).
Same problem with Descartes 1.1.
Do the problem persist in the latest version?
Just tested with descartes 1.2 (and pitest 1.4.0) with the command line: mvn clean install -Pquality -Dxwiki.pitest.skip=false
And got this error:
[INFO] --- pitest-maven:1.4.0:mutationCoverage (pitest-check) @ xwiki-commons-job ---
[INFO] Found plugin : JSON report plugin
[INFO] Found plugin : Produces a JSON file reporting mutation results aggregated by method and each method classfied according to its mutation status
[INFO] Found plugin : Generates an HTML document containing classes and methods with found testing issues
[INFO] Found plugin : Default csv report plugin
[INFO] Found plugin : Default xml report plugin
[INFO] Found plugin : Default html report plugin
[INFO] Found plugin : Removes mutation in methods which are of no interest
[INFO] Found plugin : Static initializer code detector plugin
[INFO] Found plugin : Static initializer filter plugin
[INFO] Found plugin : Excluded annotations plugin
[INFO] Found plugin : Inlined finally block filter plugin
[INFO] Found plugin : Try with resources filter
[INFO] Found plugin : Implicit null check filter
[INFO] Found plugin : For each loop filter
[INFO] Found plugin : Logging calls filter
[INFO] Found plugin : Infinite for loop filter
[INFO] Found plugin : Long running iterator loop filter
[INFO] Found plugin : For loop counter filter
[INFO] Found plugin : Kotlin junk mutations filter
[INFO] Found plugin : Max mutations per class limit
[INFO] Found plugin : Equals shortcut equivalent mutant filter
[INFO] Found plugin : Trivial return vals equivalence filter
[INFO] Found plugin : Mutant export plugin
[INFO] Found shared classpath plugin : Engine for extreme mutation operators
[INFO] Found shared classpath plugin : Default mutation engine
[INFO] Found shared classpath plugin : JUnit 5 test framework support
[INFO] Found shared classpath plugin : JUnit plugin
[INFO] Found shared classpath plugin : TestNG plugin
[INFO] Adding org.pitest:pitest-junit5-plugin to SUT classpath
[INFO] Adding eu.stamp-project:descartes to SUT classpath
[INFO] Adding org.pitest:pitest to SUT classpath
[INFO] Mutating from /Users/vmassol/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-job/target/classes
[INFO] Defaulting target classes to match packages in build directory
9:43:41 AM PIT >> INFO : Verbose logging is disabled. If you encounter an problem please enable it before reporting an issue.
9:43:41 AM PIT >> INFO : MINION : objc[70102]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/bin/java (0x1032854c0) and /Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/libinstrument.dylib (0x10331
9:43:41 AM PIT >> INFO : MINION : 14e0). One of the two will be used. Which one is undefined.
9:43:41 AM PIT >> INFO : Sending 81 test classes to minion
9:43:41 AM PIT >> INFO : Sent tests to minion
9:43:41 AM PIT >> INFO : MINION : 9:43:41 AM PIT >> INFO : Checking environment
9:43:42 AM PIT >> INFO : MINION : 9:43:42 AM PIT >> INFO : Found 56 tests
9:43:42 AM PIT >> INFO : MINION : 9:43:42 AM PIT >> INFO : Dependency analysis reduced number of potential tests by 0
9:43:42 AM PIT >> INFO : 56 tests received
9:43:43 AM PIT >> INFO : MINION : 9:43:43 AM PIT >> WARNING : More threads at end of test (5) testGetJobStatusForUnexistingJob than start. (4)
9:43:43 AM PIT >> INFO : MINION : 9:43:43 AM PIT >> SEVERE : Description [testClass=org.xwiki.job.internal.DefaultJobStatusStoreTest, name=getJobStatusInOldPlace]
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.juni
9:43:43 AM PIT >> INFO : MINION : t.Assert.assertNotNull(Assert.java:712)
at org.junit.Assert.assertNotNull(Assert.java:722)
at org.xwiki.job.internal.DefaultJobStatusStoreTest.getJobStatusInOldPlace(DefaultJobStatusStoreTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Nat
9:43:43 AM PIT >> INFO : MINION : ive Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runne
9:43:43 AM PIT >> INFO : MINION : rs.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.jun
9:43:43 AM PIT >> INFO : MINION : it.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.xwiki.test.mockito.MockitoComponentMockingRule$1.evaluate(MockitoComponentMockingRule.java:
9:43:43 AM PIT >> INFO : MINION : 188)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit
9:43:43 AM PIT >> INFO : MINION : .runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
9:43:43 AM PIT >> INFO : MINION :
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at org.junit.
9:43:43 AM PIT >> INFO : MINION : vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:39)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEach
9:43:43 AM PIT >> INFO : MINION : Remaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPip
9:43:43 AM PIT >> INFO : MINION : eline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:2
9:43:43 AM PIT >> INFO : MINION : 34)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:79)
at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:70)
a
9:43:43 AM PIT >> INFO : MINION : t org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90
9:43:43 AM PIT >> INFO : MINION : )
at org.pitest.junit5.JUnit5TestUnit.execute(JUnit5TestUnit.java:88)
at org.pitest.coverage.execute.CoverageDecorator.execute(CoverageDecorator.java:50)
at org.pitest.testapi.execute.containers.UnContainer.execute(UnContainer.java:31)
at org.pitest.te
9:43:43 AM PIT >> INFO : MINION : stapi.execute.Pitest.executeTests(Pitest.java:57)
at org.pitest.testapi.execute.Pitest.run(Pitest.java:48)
at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker.java:51)
at org.pitest.coverage.execute.CoverageMinion.main(CoverageMinion.java:9
9:43:43 AM PIT >> INFO : MINION : 0)
9:43:43 AM PIT >> INFO : MINION : 9:43:43 AM PIT >> SEVERE : Description [testClass=org.xwiki.job.internal.DefaultJobStatusStoreTest, name=getJobStatusInWrongPlaceAndWithInvalidLogArgument]
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(A
9:43:43 AM PIT >> INFO : MINION : ssert.java:41)
at org.junit.Assert.assertNotNull(Assert.java:712)
at org.junit.Assert.assertNotNull(Assert.java:722)
at org.xwiki.job.internal.DefaultJobStatusStoreTest.getJobStatusInWrongPlaceAndWithInvalidLogArgument(DefaultJobStatusStoreTest.java:117
9:43:43 AM PIT >> INFO : MINION : )
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.refl
9:43:43 AM PIT >> INFO : MINION : ect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.
9:43:43 AM PIT >> INFO : MINION : invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.xwiki.test.mockito.MockitoComponentMo
9:43:43 AM PIT >> INFO : MINION : ckingRule$1.evaluate(MockitoComponentMockingRule.java:188)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.
9:43:43 AM PIT >> INFO : MINION : runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit
9:43:43 AM PIT >> INFO : MINION : .runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.r
9:43:43 AM PIT >> INFO : MINION : unner.JUnitCore.run(JUnitCore.java:115)
at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:39)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$3$1.accept(Refe
9:43:43 AM PIT >> INFO : MINION : rencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.ut
9:43:43 AM PIT >> INFO : MINION : il.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.st
9:43:43 AM PIT >> INFO : MINION : ream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:79)
at org.junit.vintage.engine.V
9:43:43 AM PIT >> INFO : MINION : intageTestEngine.execute(VintageTestEngine.java:70)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
at org.junit.platform.launche
9:43:43 AM PIT >> INFO : MINION : r.core.DefaultLauncher.execute(DefaultLauncher.java:90)
at org.pitest.junit5.JUnit5TestUnit.execute(JUnit5TestUnit.java:88)
at org.pitest.coverage.execute.CoverageDecorator.execute(CoverageDecorator.java:50)
at org.pitest.testapi.execute.containers.UnCo
9:43:43 AM PIT >> INFO : MINION : ntainer.execute(UnContainer.java:31)
at org.pitest.testapi.execute.Pitest.executeTests(Pitest.java:57)
at org.pitest.testapi.execute.Pitest.run(Pitest.java:48)
at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker.java:51)
at org.pitest.cove
9:43:43 AM PIT >> INFO : MINION : rage.execute.CoverageMinion.main(CoverageMinion.java:90)
9:43:43 AM PIT >> SEVERE : Description [testClass=org.xwiki.job.internal.DefaultJobStatusStoreTest, name=getJobStatusInOldPlace] did not pass without mutation.
9:43:43 AM PIT >> SEVERE : Description [testClass=org.xwiki.job.internal.DefaultJobStatusStoreTest, name=getJobStatusInWrongPlaceAndWithInvalidLogArgument] did not pass without mutation.
9:43:44 AM PIT >> INFO : Calculated coverage in 2 seconds.
|[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 22.949 s
[INFO] Finished at: 2018-05-28T09:43:44+02:00
[INFO] Final Memory: 54M/568M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.pitest:pitest-maven:1.4.0:mutationCoverage (pitest-check) on project xwiki-commons-job: Execution pitest-check of goal org.pitest:pitest-maven:1.4.0:mutationCoverage failed: All tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite.
[ERROR] See http://pitest.org for more details.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
vmassol ~/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-job (master *) $
This works as of revision 91897b88ebcfdf3f073c8a2d3baaacf31379324b. Issue will be closed with the next release.
Cool. Thanks Oscar!
@oscarlvp any idea when the next release will be done? I'd like to modify the xwiki pom.xml to execute Descartes on xwiki-commons-job? Thx
@vmassol it should be done by the end of the month. I'm fixing several issues before the release.
Cool I see that Descartes 1.2.6 has been released 10 minutes ago. I guess it should arrive on https://search.maven.org/search?q=g:eu.stamp-project%20AND%20a:descartes&core=gav soon :)
I retested and confirmed it works fine on commons-job. Thanks!
When executing on xwiki-commons-job I get the following failure below.
Any idea?