neo4j / graph-data-science

Source code for the Neo4j Graph Data Science library of graph algorithms.
https://neo4j.com/docs/graph-data-science/current/
Other
621 stars 160 forks source link

Label Propagation and Weakly Connected Components stuck at loading even though log says 'Finished' #204

Closed cuneyttyler closed 2 years ago

cuneyttyler commented 2 years ago

I have below queries. Even though log says 'Finished', Neo4j browser stucks at loading and doesn't show the results.

CALL gds.labelPropagation.stream( 'entityGraph', {concurrency: 4} ) YIELD nodeId, communityId RETURN nodeId, communityId

CALL gds.wcc.stream( 'entityGraph' ) YIELD nodeId, componentId RETURN nodeId, componentId

I have a graph with about 1 billion relationships and 15 million nodes. What might be the cause of this?

When I stop the execution from the browser below exception is thrown :

022-06-13 18:10:59.612+0000 DEBUG [o.n.b.v.m.BoltResponseMessageWriterV3] Encoding a fatal failure message to send. Message: FAILURE Invalid Message 'PULL Map{qid -> Long(0), n -> Long(1000)}' cannot be handled by a session in the READY state. 2022-06-13 18:10:59.613+0000 ERROR [o.n.b.r.DefaultBoltConnection] Protocol breach detected in bolt session 'bolt-2'. org.neo4j.bolt.runtime.BoltProtocolBreachFatality: Message 'PULL Map{qid -> Long(0), n -> Long(1000)}' cannot be handled by a session in the READY state. at org.neo4j.bolt.runtime.statemachine.impl.AbstractBoltStateMachine.nextState(AbstractBoltStateMachine.java:159) ~[neo4j-bolt-4.4.7.jar:4.4.7] at org.neo4j.bolt.runtime.statemachine.impl.AbstractBoltStateMachine.process(AbstractBoltStateMachine.java:102) ~[neo4j-bolt-4.4.7.jar:4.4.7] at org.neo4j.bolt.messaging.BoltRequestMessageReader.lambda$doRead$1(BoltRequestMessageReader.java:93) ~[neo4j-bolt-4.4.7.jar:4.4.7] at org.neo4j.bolt.runtime.DefaultBoltConnection.lambda$enqueue$0(DefaultBoltConnection.java:156) ~[neo4j-bolt-4.4.7.jar:4.4.7] at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatchInternal(DefaultBoltConnection.java:252) ~[neo4j-bolt-4.4.7.jar:4.4.7] at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:187) ~[neo4j-bolt-4.4.7.jar:4.4.7] at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:177) ~[neo4j-bolt-4.4.7.jar:4.4.7] at org.neo4j.bolt.runtime.scheduling.ExecutorBoltScheduler.executeBatch(ExecutorBoltScheduler.java:257) ~[neo4j-bolt-4.4.7.jar:4.4.7] at org.neo4j.bolt.runtime.scheduling.ExecutorBoltScheduler.lambda$scheduleBatchOrHandleError$3(ExecutorBoltScheduler.java:240) ~[neo4j-bolt-4.4.7.jar:4.4.7] at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.75.Final.jar:4.1.75.Final] at java.lang.Thread.run(Thread.java:829) [?:?]

s1ck commented 2 years ago

I mean .. you're streaming 15 million rows .. this might just take a while. Try using a limit in your query or run the algo in mutate mode and stream the data via Arrow (GDS enterprise).

cuneyttyler commented 2 years ago

Of course I should wait but the log says it's 'Finished'.

s1ck commented 2 years ago

The log message means that the computation of wcc / label propagation is finished. The result of a computation is a data structure holding the (10 million) results of the computation. After the computation is finished, we convert that data structure to a stream of rows which are processed by the Cypher runtime and presented to you (the caller). We can't move the "finished" log to after the result is presented to the caller, since the Cypher runtime is outside of GDS.

s1ck commented 2 years ago

Closing this since it's expected behavior. Feel free to reopen if you think we should change something about the behavior.