search

neo4j / graph-data-science

Source code for the Neo4j Graph Data Science library of graph algorithms.
https://neo4j.com/docs/graph-data-science/current/
Other
618 stars 161 forks source link

Bug: ANN similarity array out of bounds #19

Closed tomasonjo closed 4 years ago

tomasonjo commented 4 years ago

Using the example in the docs:

CREATE
  (french:Cuisine {name:'French'}),
  (italian:Cuisine {name:'Italian'}),
  (indian:Cuisine {name:'Indian'}),
  (lebanese:Cuisine {name:'Lebanese'}),
  (portuguese:Cuisine {name:'Portuguese'}),

  (zhen:Person {name: 'Zhen'}),
  (praveena:Person {name: 'Praveena'}),
  (michael:Person {name: 'Michael'}),
  (arya:Person {name: 'Arya'}),
  (karin:Person {name: 'Karin'}),

  (praveena)-[:LIKES]->(indian),
  (praveena)-[:LIKES]->(portuguese),

  (zhen)-[:LIKES]->(french),
  (zhen)-[:LIKES]->(indian),

  (michael)-[:LIKES]->(french),
  (michael)-[:LIKES]->(italian),
  (michael)-[:LIKES]->(indian),

  (arya)-[:LIKES]->(lebanese),
  (arya)-[:LIKES]->(italian),
  (arya)-[:LIKES]->(portuguese),

  (karin)-[:LIKES]->(lebanese),
  (karin)-[:LIKES]->(italian)

And running the algo:

MATCH (p:Person)-[:LIKES]->(cuisine)
 WITH {item:id(p), categories: collect(id(cuisine))} as userData
 WITH collect(userData) as data
 CALL gds.alpha.ml.ann.stream({
   nodeProjection: '*',
   relationshipProjection: '*',
   data: data,
   algorithm: 'jaccard',
   similarityCutoff: 0.1,
   concurrency: 1
 })
 YIELD item1, item2, similarity
 return gds.util.asNode(item1).name AS from, gds.util.asNode(item2).name AS to, similarity
 ORDER BY from

I get the following error:

Failed to invoke procedure gds.alpha.ml.ann.stream: Caused by: java.lang.ArrayIndexOutOfBoundsException: 1

FlorentinD commented 4 years ago

@tomasonjo We failed to reproduce the bug using gds 1.0.0 and neo4j 3.5.16. Could you provide more details, when it occurs?

tomasonjo commented 4 years ago

I am using Neo4j 3.5.14 and gds 1.1.0

FlorentinD commented 4 years ago

Are you also using Windows?

tomasonjo commented 4 years ago

Noup, Ubuntu 16.04

vnickolov commented 4 years ago

Thanks @tomasonjo it seems it's not OS related then.

We tried the Neo4j 3.5.14 + GDS 1.1.0 and the procedure completed successfully.

Wonder if you have any other data in this database, also if you're using Neo4j Desktop or going through a Driver + application?

tomasonjo commented 4 years ago

I have tried to reproduce the error on a fresh database 3.5.17 CE and also EE but couldn't reproduce I have also reproduced exact setup with docker-compose + APOC + GDS 1.1.0 + unique constraint on Person, queried via Neo4j browser and couldn't reproduce the error.

I guess it must be something weird. I'll add the stacktrace here for reference and close the issue, since we can't reproduce it.

Stacktrace:

neo4j | 2020-04-15 16:33:55.396+0000 INFO [neo4j.BoltWorker-3 [bolt] [/172.18.0.1:48838] ] LOADING neo4j | 2020-04-15 16:33:55.424+0000 WARN Computation failed 1 neo4j | java.lang.ArrayIndexOutOfBoundsException: 1 neo4j | at org.neo4j.graphalgo.core.loading.AdjacencyBuilder$CompressingPagedAdjacency.addAll(AdjacencyBuilder.java:218) neo4j | at org.neo4j.graphalgo.core.loading.RelationshipImporter.importRelationships(RelationshipImporter.java:240) neo4j | at org.neo4j.graphalgo.core.loading.RelationshipImporter.importNatural(RelationshipImporter.java:105) neo4j | at org.neo4j.graphalgo.core.loading.HugeGraphUtil$RelationshipsBuilder.flushBuffer(HugeGraphUtil.java:248) neo4j | at org.neo4j.graphalgo.core.loading.HugeGraphUtil$RelationshipsBuilder.build(HugeGraphUtil.java:231) neo4j | at org.neo4j.graphalgo.impl.similarity.ApproxNearestNeighborsAlgorithm$RelationshipImporter.buildGraphStore(ApproxNearestNeighborsAlgorithm.java:559) neo4j | at org.neo4j.graphalgo.impl.similarity.ApproxNearestNeighborsAlgorithm.similarityStream(ApproxNearestNeighborsAlgorithm.java:150) neo4j | at org.neo4j.graphalgo.impl.similarity.SimilarityAlgorithm.generateWeightedStream(SimilarityAlgorithm.java:119) neo4j | at org.neo4j.graphalgo.impl.similarity.SimilarityAlgorithm.compute(SimilarityAlgorithm.java:80) neo4j | at org.neo4j.graphalgo.impl.similarity.SimilarityAlgorithm.compute(SimilarityAlgorithm.java:33) neo4j | at org.neo4j.graphalgo.AlgoBaseProc.lambda$compute$2(AlgoBaseProc.java:383) neo4j | at org.neo4j.graphalgo.BaseProc.runWithExceptionLogging(BaseProc.java:87) neo4j | at org.neo4j.graphalgo.AlgoBaseProc.compute(AlgoBaseProc.java:379) neo4j | at org.neo4j.graphalgo.AlgoBaseProc.compute(AlgoBaseProc.java:340) neo4j | at org.neo4j.graphalgo.similarity.SimilarityProc.stream(SimilarityProc.java:53) neo4j | at org.neo4j.graphalgo.similarity.ApproxNearestNeighborsProc.annStream(ApproxNearestNeighborsProc.java:75) neo4j | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) neo4j | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) neo4j | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) neo4j | at java.lang.reflect.Method.invoke(Method.java:498) neo4j | at org.neo4j.kernel.impl.proc.ReflectiveProcedureCompiler$ReflectiveProcedure.apply(ReflectiveProcedureCompiler.java:664) neo4j | at org.neo4j.kernel.impl.proc.ProcedureRegistry.callProcedure(ProcedureRegistry.java:219) neo4j | at org.neo4j.kernel.impl.proc.Procedures.callProcedure(Procedures.java:290) neo4j | at org.neo4j.kernel.impl.newapi.AllStoreHolder.callProcedure(AllStoreHolder.java:1098) neo4j | at org.neo4j.kernel.impl.newapi.AllStoreHolder.procedureCallRead(AllStoreHolder.java:863) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.CallSupport$$anonfun$2.apply(CallSupport.scala:62) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.CallSupport$$anonfun$2.apply(CallSupport.scala:62) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.CallSupport$.callProcedure(CallSupport.scala:167) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.CallSupport$.callReadOnlyProcedure(CallSupport.scala:64) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.TransactionBoundQueryContext.callReadOnlyProcedure(TransactionBoundQueryContext.scala:974) neo4j | at org.neo4j.cypher.internal.compatibility.v3_5.ExceptionTranslatingQueryContext$$anonfun$callReadOnlyProcedure$1.apply(ExceptionTranslatingQueryContext.scala:166) neo4j | at org.neo4j.cypher.internal.compatibility.v3_5.ExceptionTranslatingQueryContext$$anonfun$callReadOnlyProcedure$1.apply(ExceptionTranslatingQueryContext.scala:166) neo4j | at org.neo4j.cypher.internal.compatibility.v3_5.ExceptionTranslationSupport$class.translateException(ExceptionTranslationSupport.scala:33) neo4j | at org.neo4j.cypher.internal.compatibility.v3_5.ExceptionTranslatingQueryContext.translateException(ExceptionTranslatingQueryContext.scala:42) neo4j | at org.neo4j.cypher.internal.compatibility.v3_5.ExceptionTranslationSupport$class.translateIterator(ExceptionTranslationSupport.scala:47) neo4j | at org.neo4j.cypher.internal.compatibility.v3_5.ExceptionTranslatingQueryContext.translateIterator(ExceptionTranslatingQueryContext.scala:42) neo4j | at org.neo4j.cypher.internal.compatibility.v3_5.ExceptionTranslatingQueryContext.callReadOnlyProcedure(ExceptionTranslatingQueryContext.scala:166) neo4j | at org.neo4j.cypher.internal.runtime.LazyReadOnlyCallMode.callProcedure(ProcedureCallMode.scala:48) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.pipes.ProcedureCallPipe.org$neo4j$cypher$internal$runtime$interpreted$pipes$ProcedureCallPipe$$call(ProcedureCallPipe.scala:90) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.pipes.ProcedureCallPipe$$anon$$$$e6882d5096c8b8441ac36d3c0cc459e$$$$ateResultsByAppending$1.apply(ProcedureCallPipe.scala:73) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.pipes.ProcedureCallPipe$$anon$$$$e6882d5096c8b8441ac36d3c0cc459e$$$$ateResultsByAppending$1.apply(ProcedureCallPipe.scala:71) neo4j | at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:435) neo4j | at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:441) neo4j | at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409) neo4j | at scala.collection.Iterator$class.foreach(Iterator.scala:891) neo4j | at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) neo4j | at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59) neo4j | at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104) neo4j | at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48) neo4j | at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310) neo4j | at scala.collection.AbstractIterator.to(Iterator.scala:1334) neo4j | at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302) neo4j | at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1334) neo4j | at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289) neo4j | at scala.collection.AbstractIterator.toArray(Iterator.scala:1334) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.pipes.SortPipe.internalCreateResults(SortPipe.scala:36) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.pipes.PipeWithSource.createResults(Pipe.scala:76) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.pipes.PipeWithSource.createResults(Pipe.scala:72) neo4j | at org.neo4j.cypher.internal.runtime.interpreted.pipes.PipeWithSource.createResults(Pipe.scala:72) neo4j | at org.neo4j.cypher.internal.compatibility.v3_5.runtime.executionplan.BaseExecutionResultBuilderFactory$BaseExecutionWorkflowBuilder.build(DefaultExecutionResultBuilderFactory.scala:61) neo4j | at org.neo4j.cypher.internal.compatibility.InterpretedRuntime$InterpretedExecutionPlan.run(InterpretedRuntime.scala:84) neo4j | at org.neo4j.cypher.internal.compatibility.CypherCurrentCompiler$CypherExecutableQuery$$anonfun$execute$3.apply(CypherCurrentCompiler.scala:204) neo4j | at org.neo4j.cypher.internal.compatibility.CypherCurrentCompiler$CypherExecutableQuery$$anonfun$execute$3.apply(CypherCurrentCompiler.scala:190) neo4j | at org.neo4j.cypher.exceptionHandler$runSafely$.apply(exceptionHandler.scala:89) neo4j | at org.neo4j.cypher.internal.compatibility.CypherCurrentCompiler$CypherExecutableQuery.execute(CypherCurrentCompiler.scala:223) neo4j | at org.neo4j.cypher.internal.ExecutionEngine.execute(ExecutionEngine.scala:101) neo4j | at org.neo4j.cypher.internal.javacompat.ExecutionEngine.executeQuery(ExecutionEngine.java:85) neo4j | at org.neo4j.bolt.v1.runtime.TransactionStateMachineV1SPI$BoltResultHandleV1.start(TransactionStateMachineV1SPI.java:189) neo4j | at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State.startExecution(TransactionStateMachine.java:506) neo4j | at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State$1.execute(TransactionStateMachine.java:300) neo4j | at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State$1.run(TransactionStateMachine.java:269) neo4j | at org.neo4j.bolt.v1.runtime.TransactionStateMachine.run(TransactionStateMachine.java:110) neo4j | at org.neo4j.bolt.v3.runtime.ReadyState.processRunMessage(ReadyState.java:85) neo4j | at org.neo4j.bolt.v3.runtime.ReadyState.processUnsafe(ReadyState.java:56) neo4j | at org.neo4j.bolt.v3.runtime.FailSafeBoltStateMachineState.process(FailSafeBoltStateMachineState.java:48) neo4j | at org.neo4j.bolt.v1.runtime.BoltStateMachineV1.nextState(BoltStateMachineV1.java:144) neo4j | at org.neo4j.bolt.v1.runtime.BoltStateMachineV1.process(BoltStateMachineV1.java:92) neo4j | at org.neo4j.bolt.messaging.BoltRequestMessageReader.lambda$doRead$1(BoltRequestMessageReader.java:89) neo4j | at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:191) neo4j | at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:139) neo4j | at org.neo4j.bolt.runtime.ExecutorBoltScheduler.executeBatch(ExecutorBoltScheduler.java:171) neo4j | at org.neo4j.bolt.runtime.ExecutorBoltScheduler.lambda$scheduleBatchOrHandleError$2(ExecutorBoltScheduler.java:154) neo4j | at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) neo4j | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) neo4j | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) neo4j | at java.lang.Thread.run(Thread.java:748)