search

yzslab / gaussian-splatting-lightning

A 3D Gaussian Splatting framework with various derived algorithms and an interactive web viewer
Other
547 stars 45 forks source link

LightGaussian based ply model prune #78

Open hardikdava opened 1 week ago

hardikdava commented 1 week ago

hello @yzslab :wave: , is it possible to do only pruning using light-gaussian method for existing ply gaussian splatting models?

yzslab commented 1 week ago

I think you can start a new training via the option --model.initialize_from=YOUR_PLY_FILE_PATH with some adjusted hyper parameters, i.e. enabling LightGaussian, smaller the prune steps and max steps and LRs, disabling densification.

alpergel commented 1 day ago

Hi! Can you advise how I can do LightGaussian pruning on the output of the depth regularized gaussian?

yzslab commented 1 day ago

Hi! Can you advise how I can do LightGaussian pruning on the output of the depth regularized gaussian?

Try

python main.py fit \
    --config YOUR_DEPTH_REG_CONFIG \
    --config light_gaussian/prune_finetune.yaml \
    --ckpt_path last \
    ...
alpergel commented 1 day ago

Perfect thanks!

alpergel commented 1 day ago

Problem with given class_path 'internal.models.vanilla_gaussian.VanillaGaussian': 'Configuration check failed :: No action for destination key "optimization.position_lr_max_steps" to check its value.' I got this error.

yzslab commented 1 day ago

Problem with given class_path 'internal.models.vanilla_gaussian.VanillaGaussian': 'Configuration check failed :: No action for destination key "optimization.position_lr_max_steps" to check its value.' I got this error.

I forgot to update that file since the last refactoring. I will update it later.

alpergel commented 1 day ago

Gotcha, can you give me a ping please when you do? Also I will likely have the PR for TamingGS done within the week, thank you for all of your efforts!

yzslab commented 1 day ago

Gotcha, can you give me a ping please when you do? Also I will likely have the PR for TamingGS done within the week, thank you for all of your efforts!

Hi, pull the latest version and try again.

alpergel commented 1 day ago

AssertionError: not a checkpoint or point cloud can be found. I think its trying to run LightGaussian before running training

yzslab commented 1 day ago

AssertionError: not a checkpoint or point cloud can be found. I think its trying to run LightGaussian before running training

Do you have a training output? There is a yaml file in the output directory. Try using this yaml file as the value of the first --config option.

alpergel commented 1 day ago

What Im trying to do is automatically have LightGaussian prune right after the depth regularized training is completed. Is that functionality not there?

yzslab commented 1 day ago

What Im trying to do is automatically have LightGaussian prune right after the depth regularized training is completed. Is that functionality not there?

Try replacing the second --config with this one:

model:
  light_gaussian:
    prune_decay: 0.6
    prune_percent: 0.6
    prune_steps:
      - 16_000
      - 24_000

, and also removing the option --ckpt_path last.

yzslab commented 1 day ago

What Im trying to do is automatically have LightGaussian prune right after the depth regularized training is completed. Is that functionality not there?

Simply using light_gaussian/prune_finetune.yaml as the value of the second --config should work too. You just need to remove the option --ckpt_path last.

alpergel commented 1 day ago

I tried that, but it didnt seem to work. No errors, but nothing about LightGaussian in the logs and the final output ply is around 400 MB. Any advice?

yzslab commented 1 day ago

I tried that, but it didnt seem to work. No errors, but nothing about LightGaussian in the logs and the final output ply is around 400 MB. Any advice?

Please tell me your command and the contents of your config files.

alpergel commented 1 day ago

python main.py fit --data.path ../vggsfm/i10 --model.save_ply true --config configs/depth_regularization/estimated_inverse_depth-l1.yaml --config configs/light_gaussian/prune_finetune.yaml --data.image_on_cpu false --max_epochs 200 --data.num_workers 24 --n i10v3 was the command I just tried. Config files are all default. Im now trying your other suggestion of making a custom config with those contents, will update when done.

yzslab commented 1 day ago

I think the problem is related to the --max_epochs 200. Try removing it.

alpergel commented 1 day ago

Gotcha! Trying now, also wanted to ask, any tips for speed increases? My use case needs the training to be as fast as possible.

yzslab commented 1 day ago

any tips for speed increases? My use case needs the training to be as fast as possible.

The speed is highly related to the number of Gaussian. You can increase the densification_interval and densify_grad_threshold to reduce it. https://github.com/yzslab/gaussian-splatting-lightning/blob/17807003afccf005182ffa4e003c23ce65cbebeb/internal/density_controllers/vanilla_density_controller.py#L12-L33

hardikdava commented 1 day ago

btw I am thinking of creating a script for only pruning purpose something similar to this. This script will be used for zero shot pruning can work at anytime.

yzslab commented 1 day ago

btw I am thinking of creating a script for only pruning purpose something similar to this. This script will be used for zero shot pruning can work at anytime.

If your model is trained by this repo, pruning is quite simple, just like this: https://github.com/yzslab/gaussian-splatting-lightning/issues/78#issuecomment-2499661524. For example:

python main.py fit \
    --config outputs/lego/gsplat/lightning_logs/version_0/config.yaml \
    --config configs/light_gaussian/prune_finetune.yaml \
    --ckpt_path last