search

varunjampani / video_prop_networks

Code for our CVPR17 paper on "Video Propagation Networks"
BSD 3-Clause "New" or "Revised" License
91 stars 24 forks source link

Cuda out of memory #5

Closed flowgrow closed 6 years ago

flowgrow commented 6 years ago

Hi, I couldn't find any information on what kind of setup you used but my GTX 1080 with 8GB of Memory can only run Stage 0. Trying to run Stage 1 results in the following error message:

Check failed: error == cudaSuccess (2 vs. 0) out of memory

Did you use it with cuDNN? I tried to use it (both v6 and v7) but make failed when using cuDNN. The error message was "caffe cudnn.hpp error: too few arguments in function call"

so my current setup is: Ubuntu 16.04 CUDA 8 NO cuDNN GTX 1080 - 8GB

and I am running out of memory running the following commands: python run_segmentation.py 1 4 sh train_fold.sh 1

any suggestions?

varunjampani commented 6 years ago

Hi, I used K80 and Titan Pascal Xp cards for these experiments. I think, both of these cards have 12GB of memory. For training, you can try reducing the batch size to fit into memory (line-177 in input_data_layer.py). For testing, it is surprising that 8GB of memory is not enough as I used a batch size of 3 for training with 12GB card. So, I expect that a single batch would fit into 8GB of memory. I do not have GTX1080 to test this issue. Regarding cuDNN, I think, I didn't use cuDNN in my experiments.