search

llmhyy / microbat

A feedback-based debugger for interactively recommending suspicious step in buggy program execution.
55 stars 16 forks source link

[Report] Why Some Fixed Trace Cannot Pass Test Case? #165

Open llmhyy opened 6 years ago

llmhyy commented 6 years ago

I append found case as follows:

Math | 13 | org.apache.commons.math3.optimization.fitting.PolynomialFitterTest#testLargeSample Math | 14 | org.apache.commons.math3.fitting.PolynomialFitterTest#testLargeSample Math | 38 | org.apache.commons.math.optimization.direct.BOBYQAOptimizerTest#testConstrainedRosenWithMoreInterpolationPoints Closure | 26 | com.google.javascript.jscomp.ProcessCommonJSModulesTest#testWithoutExports Mockito | 5 | org.mockitointegration.NoJUnitDependenciesTest#pure_mockito_should_not_depend_JUnit Closure | 26 | com.google.javascript.jscomp.CommandLineRunnerTest#testTransformAMDAndProcessCJS Closure | 26 | com.google.javascript.jscomp.CommandLineRunnerTest#testProcessCJS Closure | 26 | com.google.javascript.jscomp.ProcessCommonJSModulesTest#testModuleName Math | 103 | org.apache.commons.math.distribution.NormalDistributionTest#testExtremeValues

Likewise, the following is the buggy trace which can pass the test cases, just name a few:

Math | 54 | org.apache.commons.math.util.FastMathTest#checkMissingFastMathClasses Math | 55 | org.apache.commons.math.util.FastMathTest#checkMissingFastMathClasses Math | 56 | org.apache.commons.math.util.FastMathTest#checkMissingFastMathClasses Math | 57 | org.apache.commons.math.util.FastMathTest#checkMissingFastMathClasses

lylytran commented 6 years ago

[Update] These are 6 cases to check for incorrect test result, the others is due to program is forced to quite because of over-long trace so marked as failed although they are pass testcases. Math 103 org.apache.commons.math.distribution.NormalDistributionTest#testExtremeValues FALSE Closure 26 com.google.javascript.jscomp.ProcessCommonJSModulesTest#testModuleName FALSE Chart 8 org.jfree.data.time.junit.WeekTests#testConstructor TRUE Closure 26 com.google.javascript.jscomp.CommandLineRunnerTest#testProcessCJS FALSE Closure 26 com.google.javascript.jscomp.CommandLineRunnerTest#testTransformAMDAndProcessCJS FALSE Mockito 5 org.mockitointegration.NoJUnitDependenciesTest#pure_mockito_should_not_depend_JUnit FALSE