search

graphdeco-inria / gaussian-splatting

Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
Other
14.35k stars 1.86k forks source link

Error with gradients when using camera with arbitrary principal points outside of Image Boundaries #868

Open wagnva opened 4 months ago

wagnva commented 4 months ago

Hello,

I am currently trying to evaluate the use case of Gaussian Splatting for 3D-Reconstruction based on Satellite Imagery. Unfortunately the process used to approximate the satellite specific camera model into the required pinhole camera results in intrinsic camera parameters with arbitrary principal points, that can far exceed the size of the image.

One example camera intrinsics to show the large values:

After modifying the getProjectionMatrix to include the shift as described in issue #144 the initial projection of the SfM pointcloud can be seen correctly. Unfortunately the next few training iterations result in an unwanted shift of the gaussians outside of the camera field of view.

Timeseries of Training Iterations (Iteration 1 -> 5)

Has anyone made a similiar observation and/or has worked on modifying the code to handle arbitrary Principal Points correctly?

UsualLin-well commented 2 months ago

Hi, I recently encountered the same problem with processing satellite imagery, have you solved it yet?

wagnva commented 2 months ago

No unfortunately not. If you find something, please let me know.