search

wbhu / Tri-MipRF

Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fields, ICCV'23 (Oral, Best Paper Finalist)
https://wbhu.github.io/projects/Tri-MipRF
451 stars 13 forks source link

eval_img error #11

Open zhouilu opened 1 year ago

zhouilu commented 1 year ago

hi, i run lego dataset. train step is ok, bug got this error when eval_img. enc.shape is torch.Size([0, 48]), why caused this?

File "/nerfs/Tri-MipRF/trainer/trainer.py", line 184, in eval metric, rb, target = self.eval_img(data) File "/root/miniconda3/envs/neurbf/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, kwargs) File "/nerfs/Tri-MipRF/trainer/trainer.py", line 167, in eval_img rb = self.model( File "/root/miniconda3/envs/neurbf/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, *kwargs) File "/nerfs/Tri-MipRF/neural_field/model/trimipRF.py", line 86, in forward ray_indices, t_starts, t_ends = nerfacc.ray_marching( File "/root/miniconda3/envs/neurbf/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(args, kwargs) File "/root/miniconda3/envs/neurbf/lib/python3.9/site-packages/nerfacc/ray_marching.py", line 196, in ray_marching sigmas = sigma_fn(t_starts, t_ends, ray_indices) File "/nerfs/Tri-MipRF/neural_field/model/trimipRF.py", line 84, in sigma_fn return self.field.query_density(positions, level_vol)['density'] File "/nerfs/Tri-MipRF/neural_field/field/trimipRF.py", line 77, in query_density self.mlp_base(enc) File "/root/miniconda3/envs/neurbf/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/root/miniconda3/envs/neurbf/lib/python3.9/site-packages/tinycudann/modules.py", line 186, in forward output = _module_function.apply( File "/root/miniconda3/envs/neurbf/lib/python3.9/site-packages/tinycudann/modules.py", line 98, in forward native_ctx, output = native_tcnn_module.fwd(input, params) RuntimeError: /nerfs/tiny-cuda-nn/include/tiny-cuda-nn/cutlass_matmul.h:330 status failed with error Error Internal

wbhu commented 1 year ago

It seems the test ray_chunk is all sampled at empty space, did you use the default hyper-parameters? And what's your GPU type, if I can reproduce this bug I can help to address it.

zhouilu commented 1 year ago

It seems the test ray_chunk is all sampled at empty space, did you use the default hyper-parameters? And what's your GPU type, if I can reproduce this bug I can help to address it.

yes, i use the default hyper-parameters and gpu is v100, cuda version is 11.3.

AI-slam commented 6 months ago

I also meet this error when eval, could you provide some solutions to solve it.

Luh1124 commented 2 months ago

It still occurs with CUDA 11.8 on A800.