Training progress: 2%|▏ | 320/15000 [00:50<34:53, 7.01it/s, Loss=0.0183386, Pts=1000000]Traceback (most recent call last):
File "/home/jiaqi/Documents/gaussian_surfels/train.py", line 288, in
training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
File "/home/jiaqi/Documents/gaussian_surfels/train.py", line 154, in training
loss.backward()
File "/home/jiaqi/anaconda3/envs/gaussian_surfels/lib/python3.7/site-packages/torch/_tensor.py", line 396, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/jiaqi/anaconda3/envs/gaussian_surfels/lib/python3.7/site-packages/torch/autograd/init.py", line 175, in backward
allow_unreachable=True, accumulate_grad=True) # Calls into the C++ engine to run the backward pass
RuntimeError: Function _RasterizeGaussiansBackward returned an invalid gradient at index 2 - got [0, 0, 3] but expected shape compatible with [0, 16, 3]
when i tried to train on my custom data, it always occured such problem and I saw someone also had this problem but after comment the line "gaussians.mask_prune(scene.getTrainCameras(), 4)" the problem was solved but for me it didn't work, does someone know how to fix my problem?
Training progress: 2%|▏ | 320/15000 [00:50<34:53, 7.01it/s, Loss=0.0183386, Pts=1000000]Traceback (most recent call last): File "/home/jiaqi/Documents/gaussian_surfels/train.py", line 288, in
training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
File "/home/jiaqi/Documents/gaussian_surfels/train.py", line 154, in training
loss.backward()
File "/home/jiaqi/anaconda3/envs/gaussian_surfels/lib/python3.7/site-packages/torch/_tensor.py", line 396, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/jiaqi/anaconda3/envs/gaussian_surfels/lib/python3.7/site-packages/torch/autograd/init.py", line 175, in backward
allow_unreachable=True, accumulate_grad=True) # Calls into the C++ engine to run the backward pass
RuntimeError: Function _RasterizeGaussiansBackward returned an invalid gradient at index 2 - got [0, 0, 3] but expected shape compatible with [0, 16, 3]
when i tried to train on my custom data, it always occured such problem and I saw someone also had this problem but after comment the line "gaussians.mask_prune(scene.getTrainCameras(), 4)" the problem was solved but for me it didn't work, does someone know how to fix my problem?