naver / dust3r

DUSt3R: Geometric 3D Vision Made Easy
https://dust3r.europe.naverlabs.com/
Other
4.63k stars 515 forks source link

Is it possible to provide rough initial camera positions to Dust3R and get the finetuned accurate camera poses from it? #116

Open suraj-avataar opened 3 weeks ago

suraj-avataar commented 3 weeks ago

As described in the title, I would like to input coarse camera positions and get refined accurate camera poses from Dust3R. From my understanding of the init_from_known_poses function, it replaces the predicted pose.

dust3r/dust3r/cloud_opt/init_im_poses.py#L24

def init_from_known_poses(self, niter_PnP=10, min_conf_thr=3):

Instead, I would like dust3R to use this as input and fine-tune it, is there any immediate provision for such? Or would I need to change the architecture to take in coarse pose as a conditioning and fine-tune it?

I would have a similar question about Dust3R taking in sparse pixel correspondences if provided.

Thanks, Suraj