neo4j-contrib / neo4j-graph-algorithms

Efficient Graph Algorithms for Neo4j
https://github.com/neo4j/graph-data-science/
GNU General Public License v3.0
769 stars 195 forks source link

support > 2bn nodes for statistics in Huge Algorithms #887

Open jexp opened 5 years ago

jexp commented 5 years ago
CALL algo.unionFind(null, 'OBSERVED_WITH', {write:true, graph:'huge', partitionProperty:"community",concurrency:3})
      YIELD nodes, setCount, loadMillis, computeMillis, writeMillis,p1, p5,p10,p25,p50,p75,p90,p95,p100;
Failed to invoke procedure `algo.unionFind`: Caused by: com.carrotsearch.hppc.BufferAllocationException: Maximum array size exceeded for this load factor (elements: 805306369, load factor: 0.750000)

I think the error is with regards to the percentiles it couldn't provide the percentiles because it has to hold the 4 Billion clusters into 1 array inorder to do that