graphdeco-inria / gaussian-splatting

Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
Other
12.95k stars 1.63k forks source link

RuntimeError: numel: integer multiplication overflow #845

Open gwilczynski95 opened 1 month ago

gwilczynski95 commented 1 month ago

Hello. I run my custom project where I use GS rasterizer. Sometimes I encounter this problem (it happens with a random interval, usually during the second epoch, nonetheless it happens always).

Do you have any idea what is the issue here? Any tips?

23 Traceback (most recent call last):
24   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 321, in <module>
25     main()
26   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 313, in main
27     train(
28   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 182, in train
29     losses, data_to_loss = iter_pass(
30   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 59, in iter_pass
31     pred_mask = render_batch(
32   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/own_gaussian_utils.py", line 324, in render_batch
33     rendered_image, _ = rasterizer(
34   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
35     return forward_call(*input, **kwargs)
36   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/diff_gaussian_rasterization/__init__.py", line 210, in forward
37     return rasterize_gaussians(
38   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/diff_gaussian_rasterization/__init__.py", line 32, in rasterize_gaussians
39     return _RasterizeGaussians.apply(
40   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/diff_gaussian_rasterization/__init__.py", line 92, in forward
41     num_rendered, color, radii, geomBuffer, binningBuffer, imgBuffer = _C.rasterize_gaussians(*args)
42 RuntimeError: numel: integer multiplication overflow

For clarification. I didn't change the code for the rasterizer, I don't pass shs, just colors_precomp after sigmoid.

LuPaoPao commented 3 weeks ago

Hello. I run my custom project where I use GS rasterizer. Sometimes I encounter this problem (it happens with a random interval, usually during the second epoch, nonetheless it happens always).

Do you have any idea what is the issue here? Any tips?

23 Traceback (most recent call last):
24   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 321, in <module>
25     main()
26   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 313, in main
27     train(
28   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 182, in train
29     losses, data_to_loss = iter_pass(
30   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/train_cloth_rigid_nonrigid.py", line 59, in iter_pass
31     pred_mask = render_batch(
32   File "/home/olaf/grzegos/wear_it/human_mesh_work1/torch_scripts/own_gaussian_utils.py", line 324, in render_batch
33     rendered_image, _ = rasterizer(
34   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
35     return forward_call(*input, **kwargs)
36   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/diff_gaussian_rasterization/__init__.py", line 210, in forward
37     return rasterize_gaussians(
38   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/diff_gaussian_rasterization/__init__.py", line 32, in rasterize_gaussians
39     return _RasterizeGaussians.apply(
40   File "/home/olaf/miniconda3/envs/splats310/lib/python3.10/site-packages/diff_gaussian_rasterization/__init__.py", line 92, in forward
41     num_rendered, color, radii, geomBuffer, binningBuffer, imgBuffer = _C.rasterize_gaussians(*args)
42 RuntimeError: numel: integer multiplication overflow

For clarification. I didn't change the code for the rasterizer, I don't pass shs, just colors_precomp after sigmoid.

Same issues here, can you solve this now?