Open aiyb1314 opened 1 year ago
Hi it is hard to tell what could be wrong from this among of information. One possible trigger of this might be zero shape tensor being sent to tiny-cuda-nn.
I hit this as well when running train_ngp_nerf_occ.py. It always occurs after training so my suspicion is that render_image_with_occgrid_test is somehow the offender. I tested this using the nerf synthetic datasets
During the model testing phase, the following error occurs : File "/home/xuj/anaconda3/envs/nerfacc/lib/python3.10/site-packages/tinycudann-1.7-py3.10-linux-x86_64.egg/tinycudann/modules.py", line 89, in forward native_ctx, output = native_tcnn_module.fwd(input, params) RuntimeError: /home/xuj/tiny-cuda-nn/include/tiny-cuda-nn/cutlass_matmul.h:332 status failed with error Error Internal After debugging, it is found that it is caused by the number of sampling points in the test phase is 0. How to avoid this phenomenon?
I encountered the same issue, during testing traverse_grids returned an empty [].
RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with
TORCH_USE_CUDA_DSA
to enable device-side assertions.