search

Kai-46 / ARF-svox2

Artistic Radiance Fields
BSD 2-Clause "Simplified" License
498 stars 42 forks source link

Cuda out of memory after a few epochs #8

Closed LiquidAmmonia closed 1 year ago

LiquidAmmonia commented 1 year ago

Hi, I have encontered CUDA out of memory after a few successful epochs, could you tell me how to solve this?

INFO: my command line: bash try_tnt.sh Playground 1 my GPU: a Nvidia Titan RTX with 24G Memory

Bug snapshot: Snipaste_2022-08-26_18-37-25

Thank you in advance for you help!

LiquidAmmonia commented 1 year ago

P. S. This bug occured in the svox2 optimization process

Kai-46 commented 1 year ago

You might want to check the issues in the svox2 codebase for resolving this issue.

kravrolens commented 1 year ago

encountered the same problem, how did you solve this problem? thanks in advance!

Kai-46 commented 1 year ago

You could reduce the max spatial resolution in the config file for the photorealistic radiance fields reconstruction.

kravrolens commented 1 year ago

Thank you for your reply! i adjusted the hyper-params in custom.json, removed 512 resolution option in --reso, changed 1024 to 512 in --background_reso, no memory explosion occurred. But the rendering result only contains a part of the scene. don't know how to solve it, many thanks in advance!