szpak / gradle-pitest-plugin

Gradle plugin for PIT Mutation Testing
http://gradle-pitest-plugin.solidsoft.info/
211 stars 57 forks source link

Subsequent runs of the ` ./gradlew pitest` do not produce cmd output #323

Closed igorwojda closed 1 year ago

igorwojda commented 1 year ago

At the top 1st run of ./gradlew pitest - output ok At the bottom 2nd run of ./gradlew pitest - no proper output

image
szpak commented 1 year ago

Please run it with --info to confirm that Gradle treats it as "up-to-date" as there were no changes in your code, so assuming the mutation testing is stable (might not be, especially with the integration tests), there is no need to re-execute the PIT analysis (the result should be the same).

Assuming the aforementioned, if you need to re-run the PIT execution, please use --rerun-tasks (or you might also play with the up-to-date status of the pitest tasks to achieve the same just for it).

igorwojda commented 1 year ago

Works, thx

However I think this behavior is incorrect - sure from a project perspective there is no point in running this task as it is redundant, but the overall expectation tests run each time we run the Gradle command.

I suggest either running these tests or if you are sure that there when no changes just display the last (up-to-date) report on the console.

Consider a scenario where I have a few SURVIVED mutations, but I can't determine which ones survived due to lack of console output (again--rerun-tasks param works, but this contradicts JUnit default behavior where these tests just run).

C-Otto commented 1 year ago

the overall expectation tests run each time we run the Gradle command

No, which you can verify using the regular test task. Once it passes, the tests are not run again.

C-Otto commented 1 year ago

Just to make sure: you can find the report in build/reports/pitest/. If it's still there, there's no need to rerun the pitest task. If it is missing (feel free to delete the directory), the task is run again (or the result is retrieved from some build cache).

szpak commented 1 year ago

Thanks @C-Otto for fast answer. I couldn't write more. Caching is nice (and consistent with test) and reports are your friend @igorwojda.

szpak commented 1 year ago

Btw, @igorwojda Gradle 7.6 has just introduced an ability to rerun just one task, so

./gradlew pitest --rerun

should work.