search

nerfbaselines / nerfbaselines

Reproducible evaluation of NeRF and 3DGS methods
https://nerfbaselines.github.io/
MIT License
201 stars 8 forks source link

Memory problems with custom dataset #29

Closed Mikey-E closed 1 month ago

Mikey-E commented 1 month ago

For gaussian-opacity-fields it appears that grads and grads_abs may not be getting cleared correctly between calls of densify_and_prune(). They seem to double in size each time, eventually leading to out of memory errors. As such, simply changing memory parameters or environment variables does little to help. What is the best way to manage this size during training?

jkulhanek commented 1 month ago

Hi, thank you for your question. Grads and grads_abs doubling in size means that the number of gaussians doubles each densify step; am I right? From my understanding, the issue is caused by GOF, not by nerfbaselines (it calls GOF code internally). If that is the case, asking at https://github.com/autonomousvision/gaussian-opacity-fields might be better. I believe the issue can be the scale of your scene. I would recommend trying to adjust the densification and pruning thresholds. Btw, do you have the same issue with gaussian-splatting or mip-splatting?

Mikey-E commented 1 month ago

We don't seem to have it with gaussian-splatting or mip-splatting. I will ask at gaussian-opacity-fields.

jkulhanek commented 1 month ago

Closing the issue as it seems to be the issue with GOF rather then nerfbaselines.