search

spla-tam / SplaTAM

SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM (CVPR 2024)
https://spla-tam.github.io/
BSD 3-Clause "New" or "Revised" License
1.58k stars 174 forks source link

Q about pose estimation #28

Closed TwiceMao closed 11 months ago

TwiceMao commented 11 months ago

It appears that your diff-gaussian-rasterization-w-depth does not seem to return gradients with respect to camera poses during backpropagation. Is it something I overlooked, or is it intentional that you are not optimizing camera poses through gradients with respect to camera poses?

JayKarhade commented 11 months ago

Hi @TwiceMao , could you be more specific about which part of the code you noticed this, and also provide some output?

Nik-V9 commented 11 months ago

Hi, Thanks for your query!

We don't explicitly calculate gradients for the camera pose using CUDA. Instead, we use Pytorch. The camera pose can be modeled as a rigid rotation and translation of the Gaussians themselves. This is what you would do internally in a CUDA implementation anyway.

https://github.com/spla-tam/SplaTAM/blob/bbaf5cc5754bf1034b33902007872c694e412a31/utils/slam_helpers.py#L216

Doing this in Pytorch enables us to directly compute gradients through auto-differentiation as opposed to deriving the Jacobians ourselves.

TwiceMao commented 11 months ago

Nik-V9's answer is exactly what I need, thank you for your enthusiastic answer~