Bug: Lock in UnionFind on HeavyGraph (650M, 2.5BN)

https://community.neo4j.com/t/union-find-deadlock/6181

Hi,

I tried running the union find algorithm on my graph, but it seems to have hit a deadlock. It's been running for 12+ hours but the server shows no CPU usage or I/O.

I suspect the issue is related to the size of my graph. It has 650 million nodes, 2.5 billion relationships, and is about 200 GB on disk.

My graph model is very simple: there is only one node label and one relationship label

How can I resolve this deadlock? It seems most threads are stuck here:

    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
    at org.neo4j.graphalgo.impl.ParallelUnionFindQueue.lambda$compute$1(ParallelUnionFindQueue.java:85)
    at org.neo4j.graphalgo.impl.ParallelUnionFindQueue$$Lambda$450/1742932504.run(Unknown Source)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    at org.neo4j.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:122)

Cypher queries used:

// Load graph, this completes successfully
CALL algo.graph.load('my-graph','Label','REL_TYPE',{graph:'heavy'})
  YIELD name, graph, direction, undirected, sorted, nodes, loadMillis, alreadyLoaded,
        nodeWeight, relationshipWeight, nodeProperty, loadNodes, loadRelationships;

// This query hangs
CALL algo.unionFind('User', 'FRIEND', {graph:'my-graph',write:true, partitionProperty:"partition"})
YIELD nodes, setCount, loadMillis, computeMillis, writeMillis;

It seems I cannot attach text files, but I have uploaded the Java stack trace and log files here: Stack trace neo4j.log debug.log

Neo4j version: 3.5.3 Neo4j graph algorithms version: 3.5.3.3

Thanks, Steven

neo4j-contrib / neo4j-graph-algorithms

Bug: Lock in UnionFind on HeavyGraph (650M, 2.5BN) #864