search

turandai / gaussian_surfels

Implementation of the SIGGRAPH 2024 conference paper "High-quality Surface Reconstruction using Gaussian Surfels".
540 stars 26 forks source link

Cuda error in diff-gaussian-rasterizer when passing a pre-computed cov3D #21

Closed shenwenhao01 closed 5 months ago

shenwenhao01 commented 6 months ago

I passed cov3D_precomp into the rasterizer to apply a transform matrix. 

if pipe.compute_cov3D_python:              # set to True
        cov3D_precomp = pc.get_covariance(scaling_modifier, transforms.squeeze())
else:
        scales = pc.get_scaling
        rotations = pc.get_rotation

But when I use the rasterizer, it reports a cuda error.

rendered_image, rendered_normal, rendered_depth, rendered_opac, radii = rasterizer(
        means3D = means3D,
        means2D = means2D,
        shs = shs,
        colors_precomp = colors_precomp,
        opacities = opacity,
        scales = scales,
        rotations = rotations,
        cov3D_precomp = cov3D_precomp)
Traceback (most recent call last):
  File "/home/shenwenhao/anaconda3/envs/pt2/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/shenwenhao/anaconda3/envs/pt2/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "train.py", line 393, in <module>
    training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
  File "train.py", line 128, in training
    render_pkg = render(viewpoint_cam, gaussians, pipe, background, patch_size)
  File "/home/shenwenhao/GauHuman_gaussian_surfels/gaussian_renderer/__init__.py", line 144, in render
    rendered_image, rendered_normal, rendered_depth, rendered_opac, radii = rasterizer(
  File "/home/shenwenhao/anaconda3/envs/pt2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/shenwenhao/anaconda3/envs/pt2/lib/python3.8/site-packages/diff_gaussian_rasterization/__init__.py", line 234, in forward
    return rasterize_gaussians(
  File "/home/shenwenhao/anaconda3/envs/pt2/lib/python3.8/site-packages/diff_gaussian_rasterization/__init__.py", line 35, in rasterize_gaussians
    return _RasterizeGaussians.apply(
  File "/home/shenwenhao/anaconda3/envs/pt2/lib/python3.8/site-packages/torch/autograd/function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/shenwenhao/anaconda3/envs/pt2/lib/python3.8/site-packages/diff_gaussian_rasterization/__init__.py", line 104, in forward
    num_rendered, color, normal, depth, opac, radii, geomBuffer, binningBuffer, imgBuffer = _C.rasterize_gaussians(*args)
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

I was sure that all the input tensors to the rasterizer are on the same device. I wonder if there is some error in the rasterizer?

I have the pytorch2.0, python3.8.

shenwenhao01 commented 6 months ago

When I run the original program (only modify "pipe.compute_cov3D_python = True") with your suggested environment, it throws out a more detailed error:

Traceback (most recent call last):
  File "/home/shenwenhao/anaconda3/envs/gaussian_surfels/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/shenwenhao/anaconda3/envs/gaussian_surfels/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 322, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 136, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/home/shenwenhao/.vscode-server/extensions/ms-python.debugpy-2024.2.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "train.py", line 291, in <module>
    training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
  File "train.py", line 94, in training
    render_pkg = render(viewpoint_cam, gaussians, pipe, background, patch_size)
  File "/home/shenwenhao/gaussian_surfels/gaussian_renderer/__init__.py", line 105, in render
    cov3D_precomp = cov3D_precomp)
  File "/home/shenwenhao/anaconda3/envs/gaussian_surfels/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/shenwenhao/anaconda3/envs/gaussian_surfels/lib/python3.7/site-packages/diff_gaussian_rasterization/__init__.py", line 246, in forward
    raster_settings
  File "/home/shenwenhao/anaconda3/envs/gaussian_surfels/lib/python3.7/site-packages/diff_gaussian_rasterization/__init__.py", line 47, in rasterize_gaussians
    raster_settings,
  File "/home/shenwenhao/anaconda3/envs/gaussian_surfels/lib/python3.7/site-packages/diff_gaussian_rasterization/__init__.py", line 104, in forward
    num_rendered, color, normal, depth, opac, radii, geomBuffer, binningBuffer, imgBuffer = _C.rasterize_gaussians(*args)
RuntimeError: numel: integer multiplication overflow
turandai commented 6 months ago

Hi, the precomputed conv3d is inherited from the original 3DGS code, and I have never used or tested this pipeline in my modified rasterizer. I am afraid that it is not supported yet, very sorry about this.