search

spla-tam / SplaTAM

SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM (CVPR 2024)
https://spla-tam.github.io/
BSD 3-Clause "New" or "Revised" License
1.56k stars 174 forks source link

question about the first pose #131

Open mah-asd opened 1 month ago

mah-asd commented 1 month ago

Hi, thank you for the great work.

I have a question about the first pose which should be identity, if I transform the first pose in dataset, then the splatam uses that first frame pose to do tracking for the rest of frame relative to that. But by changing the first frame to a non identity matrix, the result is very bad. I think I'm missing something here.

Nik-V9 commented 1 month ago

Hi, Thanks for your interest in our work! Can you elaborate further?

I think this behavior might have to do with the stability of optimization world frame poses as opposed to relative poses.

mah-asd commented 1 month ago

It seems like even if the relative_pose in dataset is set to False, the initialization for camera poses in params and also the gaussians translations and rotations are defined to have the first pose as identity. I manually changed them according to gt_pose for first frame, but I think some where in eval function also needs some adjustment.

mah-asd commented 1 month ago

0019

This is the result when I only change relative_pose variable in loading dataset to False

Nik-V9 commented 1 month ago

Hi, I don't quite follow. This frame looks correct to me. Are you indicating that new gaussian aren't being added?

mah-asd commented 1 month ago

Thanks for your reply, how do you interpret the PSNR 16.22?

mah-asd commented 1 month ago

I mean how do I fix the poses for eval part? it seems like there is a transformation mismatch. Am I missing any step to get a rendered image without that black frame?