search

SpectacularAI / 3dgs-deblur

[ECCV2024] Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion
https://spectacularai.github.io/3dgs-deblur/
Apache License 2.0
161 stars 11 forks source link

Have you test on fast moving camera? #12

Closed StarsTesla closed 2 months ago

StarsTesla commented 3 months ago

Thanks for your contribution, I wonder if you have test this method on fast moving camera, like street scene(fast car moving will gives much more motion blur and pose noisey, especically touch aeb, or some like a huge acceleration)

oseiskar commented 2 months ago

Short answer: yes. This will not work if the blur or movement is too extreme.

Long answer: The method that is presented in the paper only modifies a certain part of the 3DGS reconstruction pipeline. How well and robustly it will work in different cases is often more dependent on the other parts. If you use this codebase and its otherwise "standard" 3DGS pipeline on very challenging data, it is likely that the COLMAP phase will fail completely and there is nothing the other parts of the pipeline can do to fix this. This is likely to happen if you film a scene from a moving. Also note that motion blur from moving objects is not compensated here, only motion blur caused by camera motion.

This method & codebase currently works for handheld mobile phone captures, which are not too challenging for COLMAP to register, but still contain a lot of blurry key frames and the overall reconstruction is more accurate with this method than plain gsplat (or at least better than the version of gsplat & Nerfstudio that this code is based on, rebasing probably further improves the performance).