search

spla-tam / SplaTAM

SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM (CVPR 2024)
https://spla-tam.github.io/
BSD 3-Clause "New" or "Revised" License
1.59k stars 174 forks source link

Queries regarding Paper Equations #42

Closed hyunskyu closed 11 months ago

hyunskyu commented 11 months ago

I have some questions about your paper

  1. In the "Method" section, µ represents the center position and it has 3 parameters (possibly x, y, z). When projecting the Gaussian into 2D pixel space, Et and µ are multiplied together. Does this mean Et is a 3 x 3 matrix containing rotation and translation? Or µ is transformed to homogeneous coordinate?

  2. Did you not use Spherical Harmonics (SH) for color representation? (Was optimization done using just the RGB 3 channels?)

  3. Is the Gaussian always isotropic because the radius r is represented in only 1 channel?

Nik-V9 commented 11 months ago

Hi, Thanks for these questions!

  1. E_t is a standard 3 x 4 extrinsic matrix. µ is transformed to homogenous coordinates as shown here: https://github.com/spla-tam/SplaTAM/blob/bbaf5cc5754bf1034b33902007872c694e412a31/utils/slam_helpers.py#L249
  2. No, we didn't use Spherical Harmonics since we didn't observe a significant change in rendering performance with SH present. As you mentioned, we just use 3 RGB channels as defined here: https://github.com/spla-tam/SplaTAM/blob/bbaf5cc5754bf1034b33902007872c694e412a31/scripts/splatam.py#L138
  3. Yes, the Gaussians are Isotropic. Please see this for more info: https://github.com/spla-tam/SplaTAM/issues/22#issuecomment-1868273289