search

spla-tam / SplaTAM

SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM (CVPR 2024)
https://spla-tam.github.io/
BSD 3-Clause "New" or "Revised" License
1.35k stars 150 forks source link

Why is the projmatrix in cam the same to first frame? #118

Open Yoona12 opened 2 weeks ago

Yoona12 commented 2 weeks ago

Thanks for your great work! When i read the code, i have following question. With camera moving, the viewmatrix of camera is changing,and thus the projmatrix is changing. But in the code, with camera moving, the cam is the same to the first frame all the time.

Nik-V9 commented 1 week ago

Hi, Thanks for the interest in our work!

In SplaTAM, we fix the viewing camera to the first frame (this is the world frame). For subsequent cameras, rather than defining a new viewing camera, we project the Gaussians to the first frame (that is the world frame). Hence, we don't need to change the viewing camera for the rasterizer.

Please refer to the following comments and let me know if this answers your query:

  1. https://github.com/spla-tam/SplaTAM/issues/52#issuecomment-1885255397
  2. https://github.com/spla-tam/SplaTAM/issues/28#issuecomment-1855166250
DeepDuke commented 1 week ago

Hi, I have the same question. Before rendering, the Gaussians are transformed to the current frame. If the viewing matrix is fixed to the first frame, shouldn't we render the transformed Gaussians in the current frame? It looks strange to render at the first frame view. Could u please help explain more, thanks a lot! @Nik-V9

DeepDuke commented 1 week ago

Another question is about calculating the loss. If the rendered images are viewed from the first frame, how do u compute the loss with ground truth images that are observed at different frames?