search

sanity / quickml

A fast and easy to use decision tree learner in java
http://quickml.org/
GNU Lesser General Public License v3.0
231 stars 54 forks source link

DecisionOldOldTreeBuilderTest::randomForestTest is also flaky #153

Closed owolabileg closed 6 years ago

owolabileg commented 6 years ago

DecisionOldOldTreeBuilderTest::randomForestTest failed for us on commit 18b60b59:

randomForestTest on randomForestTest(quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest)(quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest)  Time elapsed: 5.291 sec  <<< FAILURE!
java.lang.AssertionError: attribute: timeOfArrival-minuteOfDay.  branch.getProbabilityOfTrueChild(): 0.9
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest$Conditions.satisfiesBranchConditions(DecisionOldOldTreeBuilderTest.java:159)
        at quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest$Conditions.satisfiesConditions(DecisionOldOldTreeBuilderTest.java:152)
        at quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest.recurseTree(DecisionOldOldTreeBuilderTest.java:127)
        at quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest.recurseTree(DecisionOldOldTreeBuilderTest.java:130)
        at quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest.recurseTree(DecisionOldOldTreeBuilderTest.java:129)
        at quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest.recurseTree(DecisionOldOldTreeBuilderTest.java:129)
        at quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest.recurseTree(DecisionOldOldTreeBuilderTest.java:130)
        at quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest.recurseTree(DecisionOldOldTreeBuilderTest.java:130)
        at quickml.supervised.tree.decisionTree.DecisionOldOldTreeBuilderTest.randomForestTest(DecisionOldOldTreeBuilderTest.java:116)

We discovered this test to be flaky while using DeFlaker, and we found some races using RV-Predict.

Our initial debugging shows that this failure is due to data races (rather than random numbers). For example, there is a data race on the field, quickml.supervised.tree.branchingConditions.StandardBranchingConditions.minSplitFraction, with a concurrent read at quickml.supervised.tree.branchingConditions.StandardBranchingConditions.copy(StandardBranchingConditions.java:114) and a concurrent write at quickml.supervised.tree.branchingConditions.StandardBranchingConditions.update(StandardBranchingConditions.java:107). There is also a data race on the field, quickml.supervised.tree.branchingConditions.StandardBranchingConditions.minScore, with two concurrent writes at quickml.supervised.tree.branchingConditions.StandardBranchingConditions.update(StandardBranchingConditions.java:103) from two different threads.

Let us know if you need more help in debugging.