search

kerighan / graph-walker

Fastest random walks generator on networkx graphs
64 stars 12 forks source link

Any idea with command filed (exit code 1) #15

Open TheAllUnknown opened 2 months ago

TheAllUnknown commented 2 months ago

Hi, I am using the .random_walks working with a sparse network with 1.6 million nodes, it works well in the beginning when I have a small number of steps and walks. However, when I increased the calculation, the program failed without giving any error message but only a command filed (exit code 1)

start_nodes = np.random.choice(G.nodes(),800)
X = walker.random_walks(G, n_walks=6, walk_len=400, alpha=0.15,start_nodes=start_nodes)

Then I tried to split the random_walks

for i in range(4):
    start_nodes = np.random.choice(G.nodes(),200)
    X = walker.random_walks(G, n_walks=6, walk_len=400, alpha=0.15,start_nodes=start_nodes)

I found the first call can work but the following calls will fail like the previous. each call only takes 8-9 seconds so I think it's not related to memory issues.

Any advice would be valuable, thanks!

kerighan commented 2 months ago

Interesting, it's the first time I see this issue.

  1. Check that your RAM isn't exploding
  2. Could you try and isolate a special sample of nodes that fail? Or does it look random to you?
  3. If your data isn't sensitive, you could also share a smaller subgraph of your graph in which this issue arises

Btw, walks of such lengths would converge towards the pagerank distribution and you would likely end up visiting the most central nodes in the network and all walks would look the same eventually, which (depending on your use case) might not be that pertinent.