pitest 1.1.10 works with jmockit 1.25, does not work with jmockit 1.26

[estekhin]

A simple project with a SINGLE test class works with jmockit 1.25 and 1.26.

A simple project with TWO (or more) test classes works with jmockit 1.25 and does not work with jmockit 1.26. The exception which causes minions to fail is:

org.pitest.util.PitError: Duplicate application of the same mock-up class
        at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
        at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker.java:55)
        at org.pitest.coverage.execute.CoverageMinion.main(CoverageMinion.java:85)
Caused by: java.lang.IllegalStateException: Duplicate application of the same mock-up class
        at mockit.MockUp.findPreviouslyMockedClassIfMockUpAlreadyApplied(MockUp.java:165)
        at mockit.MockUp.<init>(MockUp.java:119)
        at mockit.integration.testng.TestNGRunnerDecorator$MockParameter
        at mockit.integration.testng.TestNGRunnerDecorator.onExecutionStart(TestNGRunnerDecorator.java:349)
        at org.testng.TestNG.runExecutionListeners(TestNG.java:1110)
        at org.testng.TestNG.run(TestNG.java:1048)
        at org.pitest.testng.TestNGTestUnit.executeInCurrentLoader(TestNGTestUnit.java:98)
        at org.pitest.testng.TestNGTestUnit.execute(TestNGTestUnit.java:68)
        at org.pitest.coverage.execute.CoverageDecorator.execute(CoverageDecorator.java:50)
        at org.pitest.testapi.execute.containers.UnContainer.execute(UnContainer.java:32)
        at org.pitest.testapi.execute.Pitest.executeTests(Pitest.java:59)
        at org.pitest.testapi.execute.Pitest.run(Pitest.java:50)
        at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker
        ... 1 more

The tests are otherwise good and green (if invoked as tests, without pitest) . The tests do not even need to use jmockit, simply having two or more test classes and a dependency to jmockit causes this problem.

The exception seems to originate from https://github.com/jmockit/jmockit1/blob/master/main/src/mockit/MockUp.java#L173. The change that introduced that behavior is https://github.com/jmockit/jmockit1/commit/00e2caaeee53f4e6cca66b5359ac73d6ed9f691d#diff-35ee262c29b80f8f866ea1d54377ae52 and it is linked to https://github.com/jmockit/jmockit1/issues/313.

[craftey]

I get the same exception when running my test-suite with infinitest plugin in IntelliJ. Version 1.25 does not have the problem. Version 1.26 produces the exception in line MockUp.java:165. Version 1.28 produces the exception in line MockUp.java:174.

[craftey]

running my test-suite with infinitest plugin in IntelliJ: Version 1.29 produces the exception in MockUp.java:178.

Version 1.30 does not produce the exception. Tested with 2 different test-suites. One test-suite successfully runs the tests with infinitest now. Unfortunately the other test-suite produces many StackOverflowError in different places:

mockit.internal.state.TestRun:147 - StackOverflowError() - 14 tests in my suite fail with this exception org.testng.internal.Parameters:-1 - StackOverflowError() - 4 tests fail like that java.lang.ThreadLocal:419 - StackOverflowError() - 1 test java.util.concurrent.ConcurentHashMap:936 - StackOverflowError() - 1 test

Running with mvn test or with the standard IntelliJ test runner works. Configuring higher stack sizes for infinitest does not help, it just takes longer until the tests fail. Playing arround with different Stacksizes -Xss1m or -Xss100m (configured in file infinitest.args) I figured, that in consecutive runs I get sometimes other Errors, meaning in the list above some errors were replaced by other errors: mockit.internal.utilClassLoad:37 - StackOverflowError() - 1 test sun.reflect.UnsafeFieldAccessorImpl:57 - StackOverflowError() - 1 test mockit.integration.testng.TestNGRunnerDecorator:48 - StackOverflowError() - 1 test

[craftey]

I added the report for my issue to jmockit tracker: https://github.com/jmockit/jmockit1/issues/379 Unfortunatelly it got closed with no comment. tbc ...

[StefanPenndorf]

Hi @craftey

I don't understand how this is related to pitest. Can you elaborate a little bit more on your setup and your problem at hand?

The StackOverflowError is usually a hint that there is an infinite loop in the code. Your findings that increasing the stack size does not help supports that theory. Maybe the infinite loop spans across several classes. Analyse the stack trace(s) of your failing test(s) to find the (common) loop.

[craftey]

Hi @KyleRogers, thanks for your kind reply.

I develop different web projects which use jMockit in their TestNG test-suites. From time to time we update the dependencies in the pom. This includes jmockit. I found, with the upgrade from jMockit 1.25 to 1.26 that the IntelliJ plugin infinitest stopped running the tests successfully in my current two projects and these are the only ones I tested.

I do not have a full stack-trace since I only have the GUI feedback of the plugin. This feedback gave me one line of the stacktrace for each failing test. Something like:

mockit.MockUp:165 - IllegalStateException(Duplicate application of the same mock-up class)

for all tests in the test-suite.

That is what I used for googling. I stumbled over this thread. To be clear, mvn test works as expected. Only running the tests with infinitest fails. So there is a specialty how infinitest runs tests. Maybe this is what pitest and infinitest have in common.

Upgrading jmockit further to 1.30 changes the behavior of running the tests with infinitest, just like I described before, a good portion of tests fail with StackOverflowError, including test classes that do not have any jMockit imports. This is in contrast to infinitest + jMockit 1.26-1.29 where all tests fail with IllegalStateException in mockit.MockUp.

mvn test works. I suspect jMockit introduced something new that breaks infinitest and maybe pitest. At least in one of my test-suits the Stackoverflow issue persists with jMockit 1.30. How does 1.30 behave with pitest examples?

[craftey]

I found a really weird workaround: https://github.com/jmockit/jmockit1/issues/379#issuecomment-271932063

[craftey]

I found the workarround does not work its making things even worse. https://github.com/jmockit/jmockit1/issues/379#issuecomment-272413066