search

snap-stanford / snap

Stanford Network Analysis Platform (SNAP) is a general purpose network analysis and graph mining library.
Other
2.18k stars 799 forks source link

bad_alloc() error when loading graph into memory #169

Closed sabetAI closed 4 years ago

sabetAI commented 5 years ago

I'm running node2vec on the wikipedia graph, but run into memory errors when node2vec tries to load the entire graph into memory. Why does the graph consume SO MUCH memory, and how can I avoid loading the entire thing into RAM all at once?

etorf commented 4 years ago

Were you able to get to a resolution? I am getting the same error, trying to load a 700M edges Graph. The graph file is only 13GB and I am using a huge RAM machine.

roks commented 4 years ago

node2vec caches intermediate results to produce the output in a reasonable time, which is very memory intensive, especially if there are many high degree nodes. A solution is to run the program on smaller subgraphs of the original graph.