Increasing GPU memory during training time: Cuda out for memory error

graspnet / graspnet-baseline

Baseline model for "GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping" (CVPR 2020)

https://graspnet.net/

Other

475 stars 142 forks source link

Increasing GPU memory during training time: Cuda out for memory error #97

Closed lishiyu005 closed 5 months ago

lishiyu005 commented 7 months ago

Dear authors,

Thanks for your great work. When I try to train the models, the GPU memory increase from 10 GB in the beginning to more than 20GB after 300 batches. How did you solve the problems? Is there any way to fix this issue?

Best regards, Shiyu Li

chenxi-wang commented 6 months ago

Hi, if you met cuda OOM, you can try sampling the points or reducing the training data.