pitest / pitest-junit5-plugin

JUnit 5 test framework support for Pitest
Apache License 2.0
74 stars 27 forks source link

Pitest failing after JUnit5 migration #60

Closed harmeshchoubisa closed 3 years ago

harmeshchoubisa commented 3 years ago

It appears after JUnit5 migration integration test are failing. It was working before with JUnit 4.

Below is the configuration in pom

<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.6.8</version>
<dependencies>
<dependency>
<groupId>org.pitest</groupId>
<artifactId>pitest-junit5-plugin</artifactId>
<version>0.14</version>
</dependency>
</dependencies>
<configuration>
<targetClasses>
<param>com.abc.def.*</param>
</targetClasses>
<targetTests>
<param>com.abc.def.*</param>
</targetTests>
<timestampedReports>false</timestampedReports>
<threads>4</threads>
<outputFormats>
<outputFormat>XML</outputFormat>
<outputFormat>HTML</outputFormat>
</outputFormats>
</configuration>
</plugin>

TestSnippet:

@Test
public void getAllServices() {
givenApplicationStarted()
.when()
.contentType(APPLICATION_JSON)
.get("/context-root/services")
.then()
.statusCode(200);

Error logs

8:56:53 PM PIT >> SEVERE : Description [testClass=com.abc.def.MyIntegrationTestIT, name=getAllServices()] 8:56:53 PM PIT >> INFO : MINION : java.lang.AssertionError: 1 expectation failed. 8:56:53 PM PIT >> INFO : MINION : Expected status code <200> but was <500>.

Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-cli of goal org.pitest:pitest-maven:1.6.8:mutationCoverage failed: 3 tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite. See http://pitest.org for more details. at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:148) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject

hcoles commented 3 years ago

If you can put together a minimal project that reproduces the problem I can take a look.

harmeshchoubisa commented 3 years ago

It would be very time consuming & difficult to put a minimal project since the issue is in the integration test that is building successfully in mvn clean install but failing only on pi test run and to simulate this we have to create an end to end flow that consist multiple modules. Unfortunately for the security reasons I cannot put the same project.

I can provide all details whatever you need or a full stacktrace by enabling verbose or specific version of the dependencies if that would be helpful.

hcoles commented 3 years ago

For some reason the code responding to the http get is erroring with a 500. To understand why we would need to see how the test starts the code under test, what the code under test is doing, exactly how the test is implelemented etc etc etc.

Producing a minimal project may be time consuming, but trying to work out what is happening without one is most likely impossible, or at the very least would take far longer.

https://stackoverflow.com/help/minimal-reproducible-example

hcoles commented 3 years ago

closing as insufficient information to reproduce