search

turandai / gaussian_surfels

Implementation of the SIGGRAPH 2024 conference paper "High-quality Surface Reconstruction using Gaussian Surfels".
527 stars 25 forks source link

Question about camera pose optimization in `cameras.py` #43

Closed Lee-JaeWon closed 1 month ago

Lee-JaeWon commented 2 months ago

Thanks for the good work.

I noticed that there is a commented camera pose optimization code in cameras.py, does it work?

I'm going to try to make it work, but how can I make sure it's optimized well?

Thanks in advance for your kind response.

turandai commented 2 months ago

Hi, I have implemented the gradients w.r.t. camera extrinsics in cuda. However, this has not yet been fully tested, I am not sure about its correctness. You may check or modify it in you own experiments.

Lee-JaeWon commented 2 months ago

@turandai I've tried it and don't see any significant impact.

Is there anything else I should consider, and in what situations have you had success?

turandai commented 2 months ago

I tried on DTU, with random noise added to the GT poses. The results are better with pose optimization than without, but still worse than using the GT poses. I am not sure about the problem but I think maybe the per-image sampling strategy of the 3dgs rasterizer is not good for pose optimization? You might check other works focusing on pose optimization for more insights.