unclebob / fitnesse

FitNesse -- The Acceptance Test Wiki
fitnesse.org
Other
2.04k stars 714 forks source link

Attempting to build the pitest target fails due to failing tests #970

Open hansjoachim opened 8 years ago

hansjoachim commented 8 years ago

When attempting to run ./gradlew pitest it fails with the following error message: Exception in thread "main" org.pitest.help.PitHelpError: All tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite.

Wasn't really obvious which test(s) failed, but with some grepping I found the following buried in the output: 10:02:32 PM PIT >> WARNING : Description [testClass=fitnesse.junit.FitNesseSuiteExampleTestNoDebug, name=FitNesse.SuiteAcceptanceTests.SuiteSlimTests.ScenarioLibraryTestSuite.ScenarioLibraryShouldIncludeIncludedPages(fitnesse.junit.FitNesseSuiteExampleTestNoDebug)] did not pass without mutation.

After ignoring this test suite, I no longer get any error messages. It seems like pitest takes forever, but it's presumably testing a lot of combinations. Don't know what the expected runtime for the complete FitNesse test suite is with pitest?

I also find it strange that pitest fails consistently on this test suite, but I haven't seen any persistent issues when running tests normally.

Anyone else able to reproduce this issue?

FitNesse: e38a89a32bebb0ec099219d58d7aab5e7b1b54e6 (current HEAD) Java: OpenJDK javac 1.8.0_102

amolenaar commented 8 years ago

Hi,

@unclebob implemented some pitest tasks in the Ant build.xml. He split up the testing per package, since a full run takes a while to complete ;). I found no way of doing that for Gradle. It should be possible, but I haven't found out how yet. I must admit I did not run a full test run either :/

On 09 Sep 2016, at 22:49, Hans Joachim Desserud notifications@github.com wrote:

When attempting to run ./gradlew pitest it fails with the following error message: Exception in thread "main" org.pitest.help.PitHelpError: All tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite.

Wasn't really obvious which test(s) failed, but with some grepping I found the following buried in the output: 10:02:32 PM PIT >> WARNING : Description [testClass=fitnesse.junit.FitNesseSuiteExampleTestNoDebug, name=FitNesse.SuiteAcceptanceTests.SuiteSlimTests.ScenarioLibraryTestSuite.ScenarioLibraryShouldIncludeIncludedPages(fitnesse.junit.FitNesseSuiteExampleTestNoDebug)] did not pass without mutation.

After ignoring this test suite, I no longer get any error messages. It seems like pitest takes forever, but it's presumably testing a lot of combinations. Don't know what the expected runtime for the complete FitNesse test suite is with pitest?

I also find it strange that pitest fails consistently on this test suite, but I haven't seen any persistent issues when running tests normally.

Anyone else able to reproduce this issue?

FitNesse: e38a89a (current HEAD Java: OpenJDK javac 1.8.0_102

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.