search

kcheng1021 / GaussianPro

[ICML2024] Official code for GaussianPro: 3D Gaussian Splatting with Progressive Propagation
https://kcheng1021.github.io/gaussianpro.github.io/
MIT License
551 stars 36 forks source link

Waymo provided poses and feature matched 3D points give far worst reconstruction results. #27

Open skrya opened 3 months ago

skrya commented 3 months ago

Firstly, I want to express my appreciation for your exceptional work.

I've been working with your model on a Waymo sequence, using the data provided via Google Drive.I have trained the model on waymo sequence with the provided data from google drive.

In the first stage I obtain the below results.json output { "ours_30000": { "SSIM": 0.9492816925048828, "PSNR": 34.894405364990234, "LPIPS": 0.23500429093837738 } }

and after second stage { "ours_30000": { "SSIM": 0.9568189978599548, "PSNR": 35.58356857299805, "LPIPS": 0.21078823506832123 }, "ours_1": { "SSIM": 0.682416558265686, "PSNR": 10.946518898010254, "LPIPS": 0.519730269908905 } }

However, when I train the same first stage by giving poses as provided in waymo and use colmap to compute the 3D points by feature matching and triangulation. I obtain far worse results for stage 1 { "ours_30000": { "SSIM": 0.8782076835632324, "PSNR": 27.07659149169922, "LPIPS": 0.277131587266922 } } Do you have any idea on why this might be happening?

kcheng1021 commented 2 months ago

Hi, thanks for your recognition. Regarding your question, here is some of my thoughts. First, have you aligned the coordinate systems of the pose you input and the point cloud? Second, 3DGS itself makes strong assumptions about camera intrinsic parameters, seemingly independent of the principal point. Is it possible that some parameters in your new input intrinsics were discarded?

skrya commented 2 months ago

I have taken care of them. However, the depth seems to be off? I notice that depth values end up between 0 to 1.2. However, for waymo provided poses the depth should be much higher (0-80). Has the depth been normalized. If yes, can you share more details how I can un-normalize it?