search

twitter / cassovary

Cassovary is a simple big graph processing library for the JVM
http://twitter.com/cassovary
Apache License 2.0
1.05k stars 150 forks source link

Exception loading graph in SharedArrayBasedDirectedGraph #188

Closed plofgren closed 9 years ago

plofgren commented 9 years ago

Today when I was loading a simple graph into SharedArrayBasedDirectedGraph, I received an ArrayIndexOutOfBoundsException. The graph is here and a minimal main class demonstrating the error is here. The stack trace is below. I just pulled the current version of cassovary, and my scala version is 2.11.6. When I change reader.toSharedArrayBasedDirectedGraph to reader.toArrayBasedDirectedGraph the graph loads correctly. I'll switch to using ArrayBasedDirectedGraph, but I wanted to report this in case someone can see what causes this and help others avoid this.

Thanks!

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 3 at com.twitter.cassovary.graph.SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphCo$$$$308510d7b33aba1419de2daaa79926$$$$nonfun$8$$anonfun$apply$15$$anonfun$apply$16$$anonfun$apply$6.apply$mcVI$sp(SharedArrayBasedDirectedGraph.scala:278) at com.twitter.cassovary.graph.SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphCo$$$$308510d7b33aba1419de2daaa79926$$$$nonfun$8$$anonfun$apply$15$$anonfun$apply$16$$anonfun$apply$6.apply(SharedArrayBasedDirectedGraph.scala:276) at com.twitter.cassovary.graph.SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphCo$$$$308510d7b33aba1419de2daaa79926$$$$nonfun$8$$anonfun$apply$15$$anonfun$apply$16$$anonfun$apply$6.apply(SharedArrayBasedDirectedGraph.scala:276) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofInt.foreach(ArrayOps.scala:234) at com.twitter.cassovary.graph.SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphConstructor$$anonfun$com$twitter$cassovary$graph$SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphConstructor$$fillInEdges$1$1$$anonfun$8$$anonfun$apply$15$$anonfun$apply$16.apply(SharedArrayBasedDirectedGraph.scala:275) at com.twitter.cassovary.graph.SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphConstructor$$anonfun$com$twitter$cassovary$graph$SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphConstructor$$fillInEdges$1$1$$anonfun$8$$anonfun$apply$15$$anonfun$apply$16.apply(SharedArrayBasedDirectedGraph.scala:274) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245) at scala.collection.Iterator$class.foreach(Iterator.scala:750) at com.twitter.cassovary.util.io.AdjacencyListGraphReader$OneShardReader$$anon$1.foreach(AdjacencyListGraphReader.scala:79) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at com.twitter.cassovary.util.io.AdjacencyListGraphReader$OneShardReader.foreach(AdjacencyListGraphReader.scala:74) at scala.collection.TraversableLike$class.map(TraversableLike.scala:245) at com.twitter.cassovary.util.io.AdjacencyListGraphReader$OneShardReader.map(AdjacencyListGraphReader.scala:74) at com.twitter.cassovary.graph.SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphConstructor$$anonfun$com$twitter$cassovary$graph$SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphConstructor$$fillInEdges$1$1$$anonfun$8$$anonfun$apply$15.apply(SharedArrayBasedDirectedGraph.scala:273) at com.twitter.cassovary.graph.SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphConstructor$$anonfun$com$twitter$cassovary$graph$SharedArrayBasedDirectedGraph$SharedArrayBasedDirectedGraphConstructor$$fillInEdges$1$1$$anonfun$8$$anonfun$apply$15.apply(SharedArrayBasedDirectedGraph.scala:273) at com.twitter.util.Try$.apply(Try.scala:13) at com.twitter.util.ExecutorServiceFuturePool$$anon$2.run(FuturePool.scala:112) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

pankajgupta commented 9 years ago

I'm looking at this

pankajgupta commented 9 years ago

Can reproduce the bug when the stored graph dir is desired to be BothInOut.

szymonm commented 9 years ago

Did you find the cause, Pankaj? If not, I can have a look at it.

On Tue, 23 Jun 2015 at 19:14 Pankaj Gupta notifications@github.com wrote:

Can reproduce the bug when the stored graph dir is desired to be BothInOut.

— Reply to this email directly or view it on GitHub https://github.com/twitter/cassovary/issues/188#issuecomment-114669360.

pankajgupta commented 9 years ago

Thanks Szymon. I didn't get a chance to get deeper yet. Please feel free to take a look

Feel free to work on branch 'issue188' that has the following two files to help reproduce/debug the error.

$ git diff --name-only master cassovary-core/src/test/resources/graphs/toy_9nodes.txt cassovary-examples/src/main/scala/HelloLoadSharedGraph.scala

On Tue, Jun 23, 2015 at 6:48 PM, Szymon notifications@github.com wrote:

Did you find the cause, Pankaj? If not, I can have a look at it.

On Tue, 23 Jun 2015 at 19:14 Pankaj Gupta notifications@github.com wrote:

Can reproduce the bug when the stored graph dir is desired to be BothInOut.

— Reply to this email directly or view it on GitHub <https://github.com/twitter/cassovary/issues/188#issuecomment-114669360 .

— Reply to this email directly or view it on GitHub https://github.com/twitter/cassovary/issues/188#issuecomment-114696484.

pankajgupta commented 9 years ago

Fixed by @szymonm