search

weify627 / 4D-Rotor-Gaussians

Apache License 2.0
79 stars 5 forks source link

illegal memory access trigged of diff_gaussian_rasterization_confidence #12

Closed florinshen closed 1 week ago

florinshen commented 2 months ago

hi, authors. thanks for your wonderful work.

when trying to reproduce the results on dnerf, i followed given steps to configure my environment. Compilation of all package passed, but when i try to run the code with following command ns-train splatfacto --data /data/dataset/DNeRF/bouncingball

an illegal memory access error is trigged in first iteration at the code line
https://github.com/weify627/4D-Rotor-Gaussians/blob/main/nerfstudio/models/splatfacto.py#L1129

RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

this seems illegal memory access existing in the differentiable rasterization package, do you have any insights about how to fix this fatal error ?

thanks for your helps in advance.

Update: with same conda environment and configuration steps, this error exists on rtx3090 gpu but disappear on a100 when reproduce results on dnerf dataset. I guess this is due to current implementation demands large shared memory size ?

weify627 commented 1 week ago

Thank you for your question! Unfortunately, we could not identify the root cause from this RuntimeError Message alone. Your guess is possible, although we didn't encounter this issue when running on RTX 2080 and RTX 3090. But it's great that you have a setting where the code can run now!