search

twitter / cassovary

Cassovary is a simple big graph processing library for the JVM
http://twitter.com/cassovary
Apache License 2.0
1.05k stars 150 forks source link

TriangleCountSpec does not consistently pass #144

Closed bmckown closed 9 years ago

bmckown commented 9 years ago

The randomly generated graph does not consistently provide a triangle count of 0 +/- 20.

https://travis-ci.org/twitter/cassovary/jobs/48922273#L1352

szymonm commented 9 years ago

you won the jackpot @bmckown

When choosing the bounds I ran the tests 1000 times. And was good (running few times). So we expect the failure probability to be < 0.001 (+/- some bias). Last results of 32 when current bound is 20 indicates that the distribution of results of the algorithm may have a high variance.

@pankajgupta, @bmckown we can: 1) increse the bounds (even further?, would that help?) 2) disable the test 3) implement sth like: when fails try again only once Any thoughts?

bmckown commented 9 years ago

Ha. Whats funny is this is actually the second time I've seen the tests fail in the same spot--once it failed locally and now in the travis-ci build.

Is part of this test to actually test the random graph generation? Could we test the triangle counting algo on a static graph of our choosing?

szymonm commented 9 years ago

No. It should test randomized triangle counting algorithm.

It needs a big graph to have good precision.

Maybe we should store sam random graph(s) to a file and use this one. Now we have two randomizations.

pankajgupta commented 9 years ago

Agree with you guys that we need to make this robust. @szymonm Is it possible to do that by choosing seed values of all random variables in the triangle counting spec and running it on a pre-stored graph

pankajgupta commented 9 years ago

I am also getting this test to fail unpredictably (e.g., in the travis ci build of https://github.com/twitter/cassovary/pull/152) . @szymonm -- would be great to make this more predictable sooner.

AnishShah commented 9 years ago

Please check this out:- TriangleCount failed. https://travis-ci.org/twitter/cassovary/jobs/51226753

szymonm commented 9 years ago

That is right. #161 is a fix