OOM when allocating tensor with shape[1289818,303]!!!!!! (during testing with pretrained network)

WeijingShi / Point-GNN

Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud, CVPR 2020.

MIT License

525 stars 114 forks source link

OOM when allocating tensor with shape[1289818,303]!!!!!! (during testing with pretrained network) #36

Closed sarimmehdi closed 3 years ago

sarimmehdi commented 3 years ago

Why is your code allocating such a large tensor? Please tell me how to fix this as my GPU cannot handle it, it doesn't make sense to have such a large tensor size for testing only.

WeijingShi commented 3 years ago

Hi @sarimmehdi , From the shape, it seems to be the features for all the edges. We put all edge features in a batch and that's usually ok when we test on GPU with 8gb memory. What's the GPU you are using? A quick workaround is to change "num_neighbors" in the config file to limit the number of edges. e.g. 256. This will impact accuracy though.

sarimmehdi commented 3 years ago

Thank you for the reply. I don't have a powerful GPU, that's why I am having trouble. I will try your code on Google Colab