search

turandai / gaussian_surfels

Implementation of the SIGGRAPH 2024 conference paper "High-quality Surface Reconstruction using Gaussian Surfels".
501 stars 24 forks source link

GPU OOM problem #18

Closed felix0903 closed 4 months ago

felix0903 commented 4 months ago

Why my GPU Memory-Usage increasing during the training progress.And it cause OOM when the training iteration is about 5k. My GPU Memory is 24G.

turandai commented 4 months ago

Hi, we use downsampled imges for warm-up training before 5k iters, the program would use more GPU memory after that. If your data contains hundreds of high-resolutin images, even 24G would be not enought. I hope this helps.

felix0903 commented 4 months ago

Thanks for answering.Will you recomand I to reduce the number of images or to decrease the image resulution by some downsample methods?

turandai commented 4 months ago

I would first recommend decreasing the resolution, and you can also try reducing the nubmer of images if your traning views are densly sampled.

felix0903 commented 4 months ago

Make sense. It's really helpful.