NVlabs / instant-ngp

Instant neural graphics primitives: lightning fast NeRF and more
https://nvlabs.github.io/instant-ngp
Other
15.78k stars 1.9k forks source link

out of memory cuda #264

Closed NoUserNameForYou closed 2 years ago

NoUserNameForYou commented 2 years ago

INFO Loading NeRF dataset from INFO data\nerf\fox\transforms.json SUCCESS Loaded 8 images of size 216x384 after 0s INFO cam_aabb=[min=[1.13457,0.406441,0.398352], max=[1.84657,1.63633,0.546498]] INFO Loading network config from: configs\nerf\base.json INFO GridEncoding: Nmin=16 b=1.66248 F=2 T=2^19 L=16 Warning: FullyFusedMLP is not supported for the selected architecture 52. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+. Warning: FullyFusedMLP is not supported for the selected architecture 52. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+. INFO Density model: 3--[HashGrid]-->32--[FullyFusedMLP(neurons=64,layers=3)]-->1 INFO Color model: 3--[Composite]-->16+16--[FullyFusedMLP(neurons=64,layers=4)]-->3 INFO total_encoding_params=13623184 total_network_params=9728 ERROR Uncaught exception: E:\instant-ngp\dependencies\tiny-cuda-nn\include\tiny-cuda-nn/gpu_memory.h:531 cuMemSetAccess(m_base_address + m_size, n_bytes_to_allocate, &access_desc, 1) failed with error CUDA_ERROR_OUT_OF_MEMORY Could not free memory: E:\instant-ngp\dependencies\tiny-cuda-nn\include\tiny-cuda-nn/gpu_memory.h:452 cuMemAddressFree(m_base_address, m_max_size) failed with error CUDA_ERROR_INVALID_VALUE

I reduced the number of images as well as their resolution and crated a new json yet I still get memory error. Tried the --width --height trick as well. I'm on gtx970 and I allwoed testbed, python and related executables with Optimize Cuda in nvcpl. Cuda 11.6 + optix 7.3 (no issues with building) + python 3.8 + win 11

metantonio commented 2 years ago

Try to down abbscale (open the json file) to 1.