search

SpoonLabs / nopol

Automatic program repair system for Java based on dynamic analysis and code synthesis with SMT. Also contains the code of Dynamoth.
https://hal.archives-ouvertes.fr/hal-01285008/document
GNU General Public License v2.0
96 stars 40 forks source link

GZoltar includes tests that are not failing in failing tests #223

Open andre15silva opened 2 years ago

andre15silva commented 2 years ago

Sometimes, GZoltar includes tests that are passing in the failing tests list:

Example cm5:

GZoltar failing test cases:

org.apache.commons.math.optimization.MultiDirectionalTest#testCostExceptions FAIL
org.apache.commons.math.optimization.MultiDirectionalTest#testRosenbrock FAIL
org.apache.commons.math.optimization.MultiDirectionalTest#testPowell FAIL
org.apache.commons.math.fraction.FractionFormatTest#testFormatImproperNegative FAIL
org.apache.commons.math.fraction.FractionFormatTest#testFormatImproper FAIL
org.apache.commons.math.fraction.FractionFormatTest#testParseProper FAIL
org.apache.commons.math.fraction.FractionFormatTest#testParseProperNegative FAIL
org.apache.commons.math.fraction.FractionFormatTest#testParse FAIL
org.apache.commons.math.fraction.FractionFormatTest#testFormatZero FAIL
org.apache.commons.math.fraction.FractionFormatTest#testFormatNegative FAIL
org.apache.commons.math.fraction.FractionFormatTest#testParseInteger FAIL
org.apache.commons.math.fraction.FractionFormatTest#testFormat FAIL
org.apache.commons.math.fraction.FractionFormatTest#testParseNegative FAIL
org.apache.commons.math.util.MathUtilsTest#testGcd_1 FAIL
org.apache.commons.math.util.MathUtilsTest#testGcd_2 FAIL
org.apache.commons.math.util.MathUtilsTest#testGcd_3 FAIL
org.apache.commons.math.util.MathUtilsTest#testGcd_4 FAIL
org.apache.commons.math.fraction.FractionTest#testFloatValue FAIL
org.apache.commons.math.fraction.FractionTest#testAbs FAIL
org.apache.commons.math.fraction.FractionTest#testAdd FAIL
org.apache.commons.math.fraction.FractionTest#testSubtract FAIL
org.apache.commons.math.fraction.FractionTest#testReciprocal FAIL
org.apache.commons.math.fraction.FractionTest#testEpsilonLimitConstructor FAIL
org.apache.commons.math.fraction.FractionTest#testGetReducedFraction FAIL
org.apache.commons.math.fraction.FractionTest#testConstructorDouble FAIL
org.apache.commons.math.fraction.FractionTest#testCompareTo FAIL
org.apache.commons.math.fraction.FractionTest#testDoubleConstructor FAIL
org.apache.commons.math.fraction.FractionTest#testLongValue FAIL
org.apache.commons.math.fraction.FractionTest#testDigitLimitConstructor FAIL
org.apache.commons.math.fraction.FractionTest#testIntValue FAIL
org.apache.commons.math.fraction.FractionTest#testDivide FAIL
org.apache.commons.math.fraction.FractionTest#testMultiply FAIL
org.apache.commons.math.fraction.FractionTest#testEqualsAndHashCode FAIL
org.apache.commons.math.fraction.FractionTest#testNegate FAIL
org.apache.commons.math.fraction.FractionTest#testGoldenRatio FAIL
org.apache.commons.math.fraction.FractionTest#testIntegerOverflow FAIL
org.apache.commons.math.fraction.FractionTest#testDoubleValue FAIL
org.apache.commons.math.fraction.FractionTest#testConstructor FAIL
org.apache.commons.math.linear.SingularValueSolverTest#testSolve FAIL
org.apache.commons.math.linear.SingularValueSolverTest#testSolveDimensionErrors FAIL
org.apache.commons.math.linear.SingularValueSolverTest#testSolveSingularityErrors FAIL
org.apache.commons.math.linear.SingularValueSolverTest#testConditionNumber FAIL
org.apache.commons.math.ode.nonstiff.AdamsMoultonIntegratorTest#testPredictorCoefficients FAIL
org.apache.commons.math.ode.nonstiff.AdamsMoultonIntegratorTest#testDimensionCheck FAIL
org.apache.commons.math.ode.nonstiff.AdamsMoultonIntegratorTest#testBackward FAIL
org.apache.commons.math.ode.nonstiff.AdamsMoultonIntegratorTest#testSmallStep FAIL
org.apache.commons.math.ode.nonstiff.AdamsMoultonIntegratorTest#testDecreasingSteps FAIL
org.apache.commons.math.ode.nonstiff.AdamsMoultonIntegratorTest#testBigStep FAIL
org.apache.commons.math.ode.nonstiff.AdamsMoultonIntegratorTest#testCorrectorCoefficients FAIL
org.apache.commons.math.linear.SingularValueDecompositionImplTest#testUOrthogonal FAIL
org.apache.commons.math.linear.SingularValueDecompositionImplTest#testMoreColumns FAIL
org.apache.commons.math.linear.SingularValueDecompositionImplTest#testDimensions FAIL
org.apache.commons.math.linear.SingularValueDecompositionImplTest#testConditionNumber FAIL
org.apache.commons.math.linear.SingularValueDecompositionImplTest#testAEqualUSVt FAIL
org.apache.commons.math.linear.SingularValueDecompositionImplTest#testMoreRows FAIL
org.apache.commons.math.linear.SingularValueDecompositionImplTest#testVOrthogonal FAIL
org.apache.commons.math.linear.SingularValueDecompositionImplTest#testMatricesValues1 FAIL
org.apache.commons.math.linear.SingularValueDecompositionImplTest#testMatricesValues2 FAIL
org.apache.commons.math.ode.nonstiff.AdamsBashforthIntegratorTest#testDimensionCheck FAIL
org.apache.commons.math.ode.nonstiff.AdamsBashforthIntegratorTest#testBackward FAIL
org.apache.commons.math.ode.nonstiff.AdamsBashforthIntegratorTest#testSmallStep FAIL
org.apache.commons.math.ode.nonstiff.AdamsBashforthIntegratorTest#testDecreasingSteps FAIL
org.apache.commons.math.ode.nonstiff.AdamsBashforthIntegratorTest#testBigStep FAIL
org.apache.commons.math.ode.nonstiff.AdamsBashforthIntegratorTest#testCoefficients FAIL

Actual failing test cases:

org.apache.commons.math.util.MathUtilsTest#testGcd_1 FAIL
org.apache.commons.math.util.MathUtilsTest#testGcd_2 FAIL
org.apache.commons.math.util.MathUtilsTest#testGcd_3 FAIL
org.apache.commons.math.util.MathUtilsTest#testGcd_4 FAIL

The same can be observed in other examples. This results in non-suspicious lines being included in the suspicious set, as well as "fuzzing" the suspiciousness values.

monperrus commented 2 years ago

interesting again!