We have observed that, in some projects, after the test cases are migrated to Junit5, the results change with no other apparent cause than the version of JUnit.
For example:
In the apache/commons-lang project, the method ReflectionDiffBuilder.accept was classified as pseudo-tested in commit apache/commons-lang@ba7ca7998f9fe0fa443e9636122944de727f9be4 but as not covered in the later commit apache/commons-lang@729adb624d3e720afb8686093814ab2bcc2d2f13
Characteristics
Description
We have observed that, in some projects, after the test cases are migrated to Junit5, the results change with no other apparent cause than the version of JUnit. For example: In the apache/commons-lang project, the method
ReflectionDiffBuilder.acceptwas classified aspseudo-testedin commit apache/commons-lang@ba7ca7998f9fe0fa443e9636122944de727f9be4 but as not covered in the later commit apache/commons-lang@729adb624d3e720afb8686093814ab2bcc2d2f13See the comparison between the two versions here
Several other methods in the same project exhibit the same behavior.
Steps to reproduce
Run PitMP for the two commits mentioned before.