search

tsattler / visloc_pseudo_gt_limitations

Other
55 stars 4 forks source link

a specific pnp implementation #4

Closed sontung closed 2 years ago

sontung commented 2 years ago

Hi,

I would like to ask what is the implementation of the PnP algorithm that you used to compute the estimated pose of your localization algorithm? I'm asking because I am having some problems with mine (correct 2d-3d matching but large differences w.r.t ground-truth poses). I am currently using openCV's solvePNPRansac. Thanks.

tsattler commented 2 years ago

We used different implementations for different methods. For Active Search, we used the P3P implementation from the examples of the RansacLib library.

ebrach commented 2 years ago

DSAC uses OpenCVs P3P as a minimal solver, and OpenCVs "iterative" (that is Levenberg-Marquardt) for refinement on all inliers. Despite using OpenCVs solvers, DSAC does not use OpenCVs RANSAC implementation, but a custom one. Not sure what solvePNPRansac does exactly. Worth checking what solvers it uses in your case, and whether it refines the estimate on all inliers. The OpenCV docs seem to say that EPNP is used by default as a minimal solver, and "iterative" as refinement. I had mixed experiences with EPNP - I found P3P to work much better in my experiments. (But not clear what solvePNPRansac does if you specify P3P as a solver flag - whether it still refines using "iterative" or not. Better check.)

sontung commented 2 years ago

I was using opencv's SQ-PNP to be exact. With this I was able to get around 96% on RedKitchen from 7scenes, however, in many bad cases, my matches are good enough. I just tried with lambda-twist pnp. With proper parameters, I was able to get 98.5%, however, the curve is not as good as the ones from other High performance algorithms. It seems to me that different PnP algorithms or implementations provide slightly different results. Is it possible that we make one default PnP solver for all the algorithms or take into account the accuracy of the matches? To analyze the accuracy of the matches, we can project 3d points using GT poses and then compute the difference between matched 2d points and projected 2d points.

tsattler commented 2 years ago

In my experience, doing proper local optimization can make quite a difference (including whether or not to use a robust cost function). I recommend using the pose estimation implementation in PoseLib (https://github.com/vlarsson/PoseLib).

sontung commented 2 years ago

I just tried PoseLib and it seemed to solve my problem. Thanks!