search

snap-stanford / ogb

Benchmark datasets, data loaders, and evaluators for graph machine learning
https://ogb.stanford.edu
MIT License
1.93k stars 397 forks source link

RuntimeError: [enforce fail at CPUAllocator.cpp:67] . DefaultCPUAllocator: can't allocate memory #140

Closed Jay7 closed 3 years ago

Jay7 commented 3 years ago

Hello, when I run the file: ogb-master/examples/lsc/mag240m/gnn.py, this error （RuntimeError: [enforce fail at CPUAllocator.cpp:67] . DefaultCPUAllocator: can't allocate memory）appears when converting the adjacency matrix（gnn.py line 61）. How should I modify the code to run on my device ？ Or is it because my device cannot complete this work? The configuration of my device is 64G RAM, four 2080ti graphics cards, 11G single-card video memory, and 16T hard disk. If it is the reason for my device, what kind of configuration is required at least to complete this work? Thank you very much! Best wishes!

rusty1s commented 3 years ago

It seems like the adj_t.to_symmetric() call will consume more than 64GB. The resulting adjacency matrix will consume about 39GB, and the intermediate process of generating reverse edges will consume some additional memory, too. I'm not sure though if there exists more memory-efficient implementations for creating undirected graphs. We tested the code with 128GB of memory which is sufficient to run most examples.

Jay7 commented 3 years ago

Thank you very much for your answers！ I will try to use a server with larger memory.