laurentkneip / opengv

OpenGV is a collection of computer vision methods for solving geometric vision problems. It is hosted and maintained by the Mobile Perception Lab of ShanghaiTech.
Other
1.01k stars 354 forks source link

KNEIP, GAO, EPNP Solvers do not return similar results to OpenCV's PnP RANSAC #121

Open alexs7 opened 1 year ago

alexs7 commented 1 year ago

@laurentkneip, I am trying to switch over to OpenGV and use some of its solvers, for absolute pose estimation.

Here is a sample of my own code: Points are in world coordinates and bearing vectors are defined as (iK * Vec3d(vp2[i].x, vp2[i].y, 1.0));

// -- OpenGV testing starts here
// create data for openGV
points_t points;
bearingVectors_t bearingVectors;
Mat33d iK = K.inv();

for (int i = 0; i < v_size; i++) {
    point_t model_point;
    model_point(0) = ap3[i].x;
    model_point(1) = ap3[i].y;
    model_point(2) = ap3[i].z;
    points.push_back(model_point);

    Vec3d iray = (iK * Vec3d(vp2[i].x, vp2[i].y, 1.0));

    bearingVector_t image_ray;
    image_ray.x() = iray(0);
    image_ray.y() = iray(1);
    image_ray.z() = iray(2);

    image_ray.normalize();
    bearingVectors.push_back(image_ray);
}

// create adapter here
absolute_pose::CentralAbsoluteAdapter adapter(bearingVectors, points);

sac::Ransac<sac_problems::absolute_pose::AbsolutePoseSacProblem> ransac;
std::shared_ptr<
sac_problems::absolute_pose::AbsolutePoseSacProblem> absposeproblem_ptr(
    new sac_problems::absolute_pose::AbsolutePoseSacProblem(
    adapter, sac_problems::absolute_pose::AbsolutePoseSacProblem::KNEIP));
ransac.sac_model_ = absposeproblem_ptr;
ransac.threshold_ = 1.0 - cos(atan(sqrt(2.0)*0.5/800.0));
ransac.max_iterations_ = 1024*2;

int64 t0 = cv::getTickCount();
ransac.computeModel();
double ransac_time = (cv::getTickCount()-t0)/cv::getTickFrequency();
//print the results
std::cout << "Ransac needed " << ransac.iterations_ << " iterations and ";
std::cout << ransac_time << " seconds" << std::endl << std::endl;
std::cout << "the number of inliers is: " << ransac.inliers_.size();
std::cout << std::endl << std::endl;
// std::cout << "the found inliers are: " << std::endl;
// for(size_t i = 0; i < ransac.inliers_.size(); i++)
//  std::cout << ransac.inliers_[i] << " ";
// std::cout << std::endl << std::endl;
std::cout << "the ransac results is: " << std::endl;
std::cout << ransac.model_coefficients_ << std::endl << std::endl;

// !-- OpenGV testing ends here

For various flags, I get different results. For example when using "KNEIP" some poses are almost identical to openCV's some poses are not. The OpenCV poses are verified to be the "ground truth". When using the "EPNP" flag I get: "God damnit, A is singular, this shouldn't happen.".

I am not sure if there is something wrong with the definition of my data (for example how I define the bearing vectors) or something else is going on ?

alexs7 commented 1 year ago

Ok it seems by relaxing the ransac error threshold to ransac.threshold_ = 2*(1.0 - cos(atan(sqrt(2.0)*0.5/800.0))); has done the trick. The results are almost identical to OpenCV now.