lvjiahui / DRT

SIGGRAPH Asia 2020: Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects
71 stars 9 forks source link

CUDA out of memory #6

Closed ChauChorHim closed 3 years ago

ChauChorHim commented 3 years ago

Hi Jiahui,

I am trying to run one of the datasets, specifically the horse. However, my local graphics card is really out-date and is not able to handle the whole dataset including 72 images. Is it possible just using half of them? I noticed that your code specifies the number of images as 72. Is there any way modifying it to adjust to 36 images? Thank you!

Here is part of the error message: RuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 3.95 GiB total capacity; 2.80 GiB already allocated; 14.94 MiB free; 2.80 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

lvjiahui commented 3 years ago

Hi ChauChorHim,

For simplicity, captured_data.py loads all 72 views data to GPU in advance. In order to load half of the refraction data, I think you only need to rewrite captured_data.py. Note that all of the silhouette data(mask) is still necessary even if you change num_view in config.py.

If CUDA is still out of memory, maybe you can try loading only what is needed per iteration.

ChauChorHim commented 3 years ago

Hi ChauChorHim,

For simplicity, captured_data.py loads all 72 views data to GPU in advance. In order to load half of the refraction data, I think you only need to rewrite captured_data.py. Note that all of the silhouette data(mask) is still necessary even if you change num_view in config.py.

If CUDA is still out of memory, maybe you can try loading only what is needed per iteration.

It works! But actually I load half of the silhouette data as well. Thanks for your reply :)