RuntimeError: CUDA error: an illegal memory access was encountered

[Liu-Sdu]

I tried to make some modifications to diff-gaussian-rasterization, adding a backpropagation. However, an error will be reported halfway through the operation. I would like to know how to open the generated snapshot_bw.dump file, and determine what the problem is?

[Liu-Sdu]

Thanks for your work

[Liu-Sdu]

There is no problem with running the original program, and there is no problem with training a specific image alone. Errors are only reported when training multiple images with different ones randomly.

[Liu-Sdu]

[CUDA ERROR] in /home/a403/LiuDon/gaussian_slam_v1/submodules/diff-gaussian-rasterization/cuda_rasterizer/rasterizer_impl.cu Line 436: an illegal memory access was encountered An error occured in backward. Writing snapshot_bw.dump for debugging.

[Liu-Sdu]

[yuedajiong]

1. unofficial answer.
2. https://docs.nvidia.com/cuda/cuda-gdb/index.html
3. CUDA thread and memory, using correctly. (global, device, auto idx=cg::this_grid().thread_rank(); you_data[idx]; .sync()，......)

[Liu-Sdu]

1. unofficial answer.
2. https://docs.nvidia.com/cuda/cuda-gdb/index.html
3. CUDA thread and memory, using correctly. (global, device, auto idx=cg::this_grid().thread_rank(); you_data[idx]; .sync()，......)

Thanks, I'll give it a try

[MilesCool]

Perhaps just ensure that all the parameters passed to gaussian_renderer is on the same device, i also got this error when i tried to modify something, finally i found some parameters is on cpu, and i changed it to cuda, everything worked.