search

hcoles / pitest

State of the art mutation testing system for the JVM
Apache License 2.0
1.69k stars 357 forks source link

Found 0 tests #767

Open Provence613 opened 4 years ago

Provence613 commented 4 years ago

I'm new to Pitest.I have a problem when I run the PIT.When I run the "mvn org.pitest: pitest-maven: mutationCoverage" command in the console, the console displays the following log：

[INFO] Adding org.pitest:pitest to SUT classpath
[INFO] Mutating from D:\iseProject\mutationSelect\newProject\commons-lang3-3.10-src\target\classes
18:37:48 PIT >> INFO : Verbose logging is disabled. If you encounter an problem please enable it before reporting an issue.
18:37:51 PIT >> INFO : Sending 754 test classes to minion
18:37:52 PIT >> INFO : Sent tests to minion
18:37:52 PIT >> INFO : MINION : 18:37:52 PIT >> INFO : Checking environment

18:37:58 PIT >> INFO : MINION : 18:37:58 PIT >> INFO : Found  0 tests

18:37:58 PIT >> INFO : MINION : 18:37:58 PIT >> INFO : Dependency analysis reduced number of potential tests by 0

18:37:58 PIT >> INFO : MINION : 18:37:58 PIT >> INFO : 0 tests received

18:37:58 PIT >> INFO : Calculated coverage in 9 seconds.
18:38:07 PIT >> WARNING : Found more than one mutation similar on same line in a finally block. Can't correct for inlining.
18:38:07 PIT >> WARNING : Found more than one mutation similar on same line in a finally block. Can't correct for inlining.
18:38:07 PIT >> WARNING : Found more than one mutation similar on same line in a finally block. Can't correct for inlining.
18:38:07 PIT >> WARNING : Found more than one mutation similar on same line in a finally block. Can't correct for inlining.
18:38:07 PIT >> WARNING : Found more than one mutation similar on same line in a finally block. Can't correct for inlining.
18:38:07 PIT >> WARNING : Found more than one mutation similar on same line in a finally block. Can't correct for inlining.
18:38:09 PIT >> INFO : Created  235 mutation test units
\18:38:47 PIT >> INFO : Completed in 59 seconds
================================================================================
- Timings
================================================================================
> scan classpath : < 1 second
> coverage and dependency analysis : 8 seconds
> build mutation tests : 11 seconds
> run mutation analysis : 37 seconds
--------------------------------------------------------------------------------
> Total  : 58 seconds
--------------------------------------------------------------------------------
================================================================================
- Statistics
================================================================================
>> Generated 13213 mutations Killed 0 (0%)
>> Ran 0 tests (0 tests per mutation)
================================================================================
- Mutators
================================================================================
> org.pitest.mutationtest.engine.gregor.mutators.ConditionalsBoundaryMutator
>> Generated 1356 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 1356
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.IncrementsMutator
>> Generated 436 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 436
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.VoidMethodCallMutator
>> Generated 970 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 970
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.InvertNegsMutator
>> Generated 20 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 20
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.ReturnValsMutator
>> Generated 4127 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 4127
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.MathMutator
>> Generated 1575 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 1575
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.NegateConditionalsMutator
>> Generated 4729 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 4729
--------------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:14 min
[INFO] Finished at: 2020-05-15T18:38:48+08:00
[INFO] ------------------------------------------------------------------------

My PIT configuration is as follows：

<plugin>
        <groupId>org.pitest</groupId>
        <artifactId>pitest-maven</artifactId>
        <version>1.3.2</version>

        <configuration>
          <targetClasses>
            <param>org.apache.commons.lang3.*</param>
          </targetClasses>
          <targetTests>
            <param>org.apache.commons.lang3.*</param>
          </targetTests>
        </configuration>
      </plugin>

How to solve this situation？

StefanPenndorf commented 4 years ago

Hi @Provence613,

I think the key to success is the usage of the JUnit 5 plugin for pitest. I'm wondering why you didn't get the

Error generating coverage. Please check that your classpath contains modern JUnit 4 or PIT test plugin for other test tool (JUnit 5, TestNG, ...) is enabled.

error message. You might have added JUnit 4 to the classpath, manually?!

But commons lang uses JUnit 5 and it seems to me you tried a recent version. I've tried it myself and created a branch in my own fork. I had to @Disable two tests in order to get it working. But those tests already seem to have special treatment for jacoco coverage and might need another hand crafted assertion when pitest coverage is in place. Afterwards pitest ran fine with 85% mutations killed. Hopefully my commit helps you to fix your own copy/project.

Please comment here if you have another problem or error instead or if this doesn't solve your problem. Please report back and close the issue if my comments solved your question.

Provence613 commented 4 years ago

Thank you for your answer. According to your prompt, I modified the version of junit, but after performing the mutation test, the mutation score is 0. What version of commons-lang are you using?

StefanPenndorf commented 4 years ago

I've created the branch based on the latest master version of commons-lang which is roughly "3.10.1-SNAPSHOT"

Do you have a fork of commons-lang yourself or can you upload your code / project to github? Then I might take a look at it. Or you could checkout my fork of commons-lang and try your experiments from there, if you like.

johannesn commented 3 years ago

Actually I do have the same issue. I set up a test project here: https://github.com/johannesn/pittestskippingtests and added a question at stackoverflow: https://stackoverflow.com/questions/66154859/pitest-skips-tests-in-ant