yihua7
commented
4 months ago Hello,
The issue is caused by the failure of node initialization. The number of control points has reduced to 0, which prevents the training from continuing.
I believe this problem is due to the initialization of control points. To resolve it, please remove the option --is_blender from your command. This option causes the nodes to be initialized from random point clouds instead of COLMAP point clouds.
Additionally, you can remove --gt_alpha_mask_as_scene_mask and add --gt_alpha_mask_as_dynamic_mask --gs_with_motion_mask if you want to model both the dynamic foreground masked by MiVOS and the static background simultaneously.
If removing --is_blender does not solve your problem, I suggest using another option: --init_isotropic_gs_with_all_colmap_pcl. This will initialize the isotropic Gaussians with all COLMAP point clouds, which can help avoid the risk of control points becoming extinct.
Thank you for your feedback, and I will update the above information in the readme file for better reference to all users.
Tyreal30416
zvict
I followed the steps of self-captured videos, then I got the input data after running phone_catch.py, but when runing train_gui.py I got the error:
train_gui.py --source_path data/bear --model_path outputs/bear --deform_type node --node_num 512 --is_blender --eval --gt_alpha_mask_as_scene_mask --local_frame --resolution 2 --W 400 --H 400 --gui Connected to pydev debugger (build 203.5981.165) Use original torch svd! Setting up [LPIPS] perceptual loss: trunk [alex], v[0.1], spatial [off] /home/zzz/anaconda3/envs/SC-GS/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead. warnings.warn( /home/zzz/anaconda3/envs/SC-GS/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or
Nonefor 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passingweights=AlexNet_Weights.IMAGENET1K_V1. You can also useweights=AlexNet_Weights.DEFAULTto get the most up-to-date weights. warnings.warn(msg) Loading model from: /home/zzz/anaconda3/envs/SC-GS/lib/python3.8/site-packages/lpips/weights/v0.1/alex.pth /home/zzz/anaconda3/envs/SC-GS/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum orNonefor 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passingweights=VGG16_Weights.IMAGENET1K_V1. You can also useweights=VGG16_Weights.DEFAULTto get the most up-to-date weights. warnings.warn(msg) Optimizing outputs/bear_node Output folder: outputs/bear_node [06/03 14:01:46] Building Learnable Gaussians for Nodes! [06/03 14:01:46] Loading trained model at iteration None [06/03 14:01:46] Reading camera 81/81 [06/03 14:01:46] Loading Training Cameras [06/03 14:01:47] Loading Test Cameras [06/03 14:02:44] Number of points at initialisation : 23529 [06/03 14:02:47] Initialize nodes with Random point cloud. [06/03 14:02:47] Initialize Learnable Gaussians for Nodes with Point Clouds! [06/03 14:02:47] Control node initialized with 512 from 23529 points. [06/03 14:02:47] Training progress: 0%| | 0/80000 [00:00<?, ?it/s]Traceback (most recent call last): File "/home/zzz/diska/projects/SC-GS/train_gui.py", line 1323, in train_node_rendering_step hyper_pcl = (trans_samp + original_gaussians.get_xyz[:, None]).reshape([original_gaussians.get_xyz.shape[0], -1]) RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1 can be any value and is ambiguous python-BaseExceptionRunning the D-NERF dataset not have this error, my input video is a static scene, does it matter？