search

turandai / gaussian_surfels

Implementation of the SIGGRAPH 2024 conference paper "High-quality Surface Reconstruction using Gaussian Surfels".
540 stars 26 forks source link

Possibility of GPU memory leak #26

Closed CanCanZeng closed 5 months ago

CanCanZeng commented 5 months ago

Hello, I found that when using render.py to calculate PSNR, the size of the video memory occupied kept increasing until OOM. In order to find the reason, I commented out all the parts after the render function https://github.com/turandai/gaussian_surfels/blob/c1cbb40b6f048e7a13535c8e0fb210c8a8192966/render.py#L39

but the video memory usage continued to increase until OOM. I suspect that there is a memory leak in the render function. I am completely unfamiliar with cuda code, especially the cuda extension of pytorch. I hope the author or others can confirm whether there is a memory leak problem, or fix the problem of video memory growing all the time.

turandai commented 5 months ago

Hi, I am not sure about the reason, but you may try add this line view.to_cpu() to the end of the render loop to move each rendered camera data out from cuda memory. The original code moves camera data to cuda one by one and does not move them out as we are testing on datasets with small numbers of views. Hope this solves your problem.

CanCanZeng commented 5 months ago

Yes, add view.to_cpu() solved my problem, many thanks!