search

SJoJoK / 3DGStream

[CVPR 2024 Highlight] Official repository for the paper "3DGStream: On-the-fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos".
https://sjojok.github.io/3dgstream
MIT License
302 stars 18 forks source link

RuntimeError: CUDA error: an illegal memory access was encountered #20

Open AnudhinaDhanabalan opened 2 months ago

AnudhinaDhanabalan commented 2 months ago

File "/home/arrstudios/Desktop/AI/3DGStream/3DGStream/gaussian_renderer/init.py", line 100, in render "visibility_filter" : radii > 0, RuntimeError: CUDA error: an illegal memory access was encountered

I am facing this error while training my own dataset, this error occurs after the training of the first frame !! thanks in advance !!

gaoyuchen2000 commented 2 months ago

I assume it is because the GPU VRAM isn't enough to satisfy the requirements. What is the GPU you are using?

AnudhinaDhanabalan commented 2 months ago

I am using nvidia rtx 6000 ada generation

WuJH2001 commented 2 months ago

I assume it is because the GPU VRAM isn't enough to satisfy the requirements. What is the GPU you are using?

I am also happended to this error. I guess it may be the error of TCNN

MarcWangzhiru commented 2 months ago

I recommend that you choose the frame where all the moving objects appear as the initial frame to initialize the 3D Gaussian distribution. I also found this problem in the process of the experiment, and later I found that when I initialized the first frame, the hand of the dynamic moving person was not present in the whole scene, but appeared in the later frames. This can lead to gradient explosion later in the optimization process. I then initialized it by picking a frame of the full dynamic scene, and the problem was solved. I suspect this is an inherent problem with 3DGStream. Can you choose the frames where the moving object appears completely as the first frame to train? I hope to test my idea.