search

facebookresearch / PoseDiffusion

[ICCV 2023] PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment
Other
718 stars 42 forks source link

Metric Pose from the reconstruction #37

Closed karanchahal closed 2 months ago

karanchahal commented 5 months ago

Hello, thanks for this great work ! I wonder how I can get netric pose from the pose estimations if I have a stereo camera and I have a known baseline between the left and right imager ?

Is this possible to add somewhere in the optimization process ?

jytime commented 5 months ago

Hi,

Sorry for the late reply. If you only care about two cameras, left and right, it should be quite easy. You just need to force the relative translation vector between these two cameras as the baseline you have, and do not optimize the translation vector. You can do it by detaching the variables corresponding to translation after this line:

https://github.com/facebookresearch/PoseDiffusion/blob/41d1cf89dc9fa8bfa134ae511bffcab84094dd83/pose_diffusion/util/geometry_guided_sampling.py#L84

karanchahal commented 5 months ago

Thanks ! Is there a way to run this on an input video ? If I give some camera intrinsics as input in a txt file (preferable colmap style)?

jytime commented 5 months ago

Hi yes you can do it although we currently do not have a util function for it. You need to replace the predicted intrinsics manually by your own (note they should be in order). A simple approach may be, first using what we have done in the demo.py to predict camera extri and intri with GGS. Then, with the predicted extri and your provided intri, do another GGS. If you are very confident about your intri, please freeze them during the second GGS optimisation.