I'm trying to build a hypergraph which has ~3.8M nodes and ~7.1M hyperedges.
My first attempt was to build a dictionary of edge -> nodelist so I could do a simple Hypergraph(dict) as per the docs, but that made no apparent progress.
I then tried to add the edges, then add the nodes. But after ~8 minutes I'm was only at 48000 edges and based on log output the process was getting markedly slower as the hypergraph grew.
I'm now trying an approach where I create a list of Entity objects then add them to an EntitySet, but that latter step has been going for many minutes now.
Skimming the code it seems like all of these bottom out in for loops that add elements, often with several uniqueness tests.
Is there an alternate construction method that I'm missing? Or am I just outside the design bounds trying to build a hypergraph this large?
I'm trying to build a hypergraph which has ~3.8M nodes and ~7.1M hyperedges.
My first attempt was to build a dictionary of edge -> nodelist so I could do a simple
Hypergraph(dict)
as per the docs, but that made no apparent progress.I then tried to add the edges, then add the nodes. But after ~8 minutes I'm was only at 48000 edges and based on log output the process was getting markedly slower as the hypergraph grew.
I'm now trying an approach where I create a list of
Entity
objects then add them to anEntitySet
, but that latter step has been going for many minutes now.Skimming the code it seems like all of these bottom out in
for
loops that add elements, often with several uniqueness tests.Is there an alternate construction method that I'm missing? Or am I just outside the design bounds trying to build a hypergraph this large?