search

Tiiiger / SGC

official implementation for the paper "Simplifying Graph Convolutional Networks"
MIT License
831 stars 145 forks source link

Memory Error #6

Closed alamsaqib closed 5 years ago

alamsaqib commented 5 years ago

While running train.py from downstream/TextSGC a Memory Error occurs. "SGC\downstream\TextSGC\utils.py", line 177, in sparse_to_torch_dense dense = sparse.todense().astype(np.int32)"

Tiiiger commented 5 years ago

May I ask which dataset you are running?

alamsaqib commented 5 years ago

Thanks for the early reply, I am using R8 and R52 under SGC\downstream\TextSGC

Tiiiger commented 5 years ago

This suggests that you don't have enough CPU memory. I can try to upload a preprocessed data so that it may take less memory.

However, in general this takes a couple of GBs memory. how many CPU memory are you using?

alamsaqib commented 5 years ago

Thanks a lot for replying. It would be great to upload less memory consumed preprocessed data. I am using 8 GB of CPU memory.

Tiiiger commented 5 years ago

I tested these scripts again. It occurs to me that the preprocessing uses much less than 8GB memory. I suspect that you might be also running other memory-intensive jobs on your machine.

That being said, I'll work on releasing a preprocessed data today.

alamsaqib commented 5 years ago

Thank you so much for early replying and really appreciate it.

Tiiiger commented 5 years ago

See https://github.com/Tiiiger/SGC/tree/master/downstream/TextSGC. Thank you for voicing this concern out. Closing for now. But please let me know if this doesn't work for you.

alamsaqib commented 5 years ago

Thank you Tiiiger for making up the changes, but it still not working for me, I am posting the error message here.

Traceback (most recent call last): File "C:\Users\Desktop\NewProjects\SGC\downstream\TextSGC\train.py", line 105, in adj_dense = sparse_to_torch_dense(sp_adj, device='cpu') File "C:\Users\Desktop\NewProjects\SGC\downstream\TextSGC\utils.py", line 127, in sparse_to_torch_dense dense = sparse.todense().astype(np.float32) File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\sparse\base.py", line 849, in todense return np.asmatrix(self.toarray(order=order, out=out)) File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\sparse\coo.py", line 317, in toarray B = self._process_toarray_args(order, out) File "C:\Users\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\sparse\base.py", line 1187, in _process_toarray_args return np.zeros(self.shape, dtype=self.dtype, order=order) MemoryError