One solution to the outrageous memory allocation causing OOM

[hubert0527]

If you are also suffering from errors similar to follows:

File "/home/???/miniconda3/envs/dgs/lib/python3.9/site-packages/diff_gaussian_rasterization/__init__.py", line 92, in forward
    num_rendered, color, depth, radii, geomBuffer, binningBuffer, imgBuffer = _C.rasterize_gaussians(*args)                                                                                              torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate **35.79 GiB**

Obviously, the 35.79 GiB memory attempted to allocate is bugged. I tested to build the same conda environment on the same dataset and the same server, and one worked while the other didn't.

Obervation Upon checking submodules/depth-diff-gaussian-rasterization/build/lib.linux-x86_64-cpython-39/diff_gaussian_rasterization/_C.cpython-39-x86_64-linux-gnu.so, I found the nvcc used different architectures for the two builds. One has: .target sm_86 The other has: .target sm_52

Also, the -gencode=arch=compute_86,code=compute_86 is missing in the failed environment during pip install when using pip install submodules/depth-diff-gaussian-rasterization --verbose.

Solution (I'm not an expert in setuptool, so this is a messy solution.) Manually add -gencode=arch=compute_86,code=compute_86 to the beginning of extra_compile_args list located in Deformable-3D-Gaussians/submodules/depth-diff-gaussian-rasterization/setup.py. It becomes:

extra_compile_args={"nvcc": ["-gencode=arch=compute_86,code=compute_86", "-I" + os.path.join(os.path.dirname(os.path.abspath(__file__)), "third_party/glm/")]})

1. compute_86 is my compute architecture, you should change it based on your GPU type.
2. Remember to delete the previous build at submodules/depth-diff-gaussian-rasterization/build/