RuntimeError: CUDA out of memory

jiapei100 commented 3 years ago

Won't 2080 Ti be able to finetune the pre-trained model?

➜  PSMNet git:(master) ✗ python finetune.py --maxdisp 192 \
                   --model stackhourglass \
                   --datatype 2015 \
                   --datapath ....../KITTI/stereo/2015/data_scene_flow/training/ \
                   --epochs 300 \
                   --loadmodel ./models/pretrained_model_KITTI2015.tar \
                   --savemodel ./
......
RuntimeError: CUDA out of memory. Tried to allocate 576.00 MiB (GPU 0; 10.76 GiB total capacity; 8.73 GiB already allocated; 301.19 MiB free; 8.74 GiB reserved in total by PyTorch)

Cheers Pei

JiaRenChang commented 3 years ago

@jiapei100 The training consumes 11.x GB per GPU and needs 4 GPUs My machine is 12GB TITANX whereas 2080ti has 11 GB. You can reduce batch size to 8.

jiapei100 commented 3 years ago

@JiaRenChang

Thank you very much for your prompt reply ... I have ONLY 1 GPU (20180Ti). Does that mean: no matter how I configure those arguments/parameters, there is NO way for me to test training PSMNet ???

Cheers

JiaRenChang commented 3 years ago

@jiapei100 You can use gradient checkpointing. https://pytorch.org/docs/stable/checkpoint.html

JiaRenChang / PSMNet

RuntimeError: CUDA out of memory #216