cdcseacave / openMVS

open Multi-View Stereo reconstruction library
http://cdcseacave.github.io
GNU Affero General Public License v3.0
3.37k stars 910 forks source link

Disparity result of opencv sgbm is different from disparity result of openMVS sgm #571

Open KeoChi opened 4 years ago

KeoChi commented 4 years ago

I try to use opencv sgbm in SemiGlobalMatcher::Match. For example:

IndexArr points;
Matrix3x3 H;
Matrix4x4 Q;
ViewData leftData, rightData;
MaskMap leftMaskMap, rightMaskMap;
{
    //TODO: use precomputed points from SelectViews()
    Point3fArr leftPoints, rightPoints;
    FOREACH(idxPoint, scene.pointcloud.points)
    {
        const PointCloud::ViewArr &views = scene.pointcloud.pointViews[idxPoint];
        if (views.FindFirst(idxImage) != PointCloud::ViewArr::NO_INDEX)
        {
            points.push_back((uint32_t)idxPoint);
            if (views.FindFirst(neighbor.idx.ID) != PointCloud::ViewArr::NO_INDEX)
            {
                const Point3 X(scene.pointcloud.points[idxPoint]);
                leftPoints.emplace_back(leftImage.camera.TransformPointW2I3(X));
                rightPoints.emplace_back(rightImage.camera.TransformPointW2I3(X));
            }
        }
    }
    // stereo-rectify image pair
    if (!Image::StereoRectifyImages(leftImage, rightImage, leftPoints, rightPoints, leftData.imageColor, rightData.imageColor, leftMaskMap, rightMaskMap, H, Q))
        continue;
    ASSERT(leftData.imageColor.size() == rightData.imageColor.size());
}
cv::Mat _disparity_map;
cv::Ptr<cv::StereoSGBM> _sgbm;
int _block_size = 3;
int _p1 = 160;//8*_block_size*_block_size;
int _p2 = 1000;//32*_block_size*_block_size;
_sgbm = cv::StereoSGBM::create(0, 368, _block_size);
_sgbm->setP1(_p1);
_sgbm->setP2(_p2);
_sgbm->setPreFilterCap(32);
_sgbm->setMinDisparity(0);
_sgbm->setNumDisparities(368);
_sgbm->setMode(cv::StereoSGBM::MODE_SGBM);
_sgbm->setUniquenessRatio(1);
_sgbm->setSpeckleWindowSize(300);
_sgbm->setSpeckleRange(1);
_sgbm->setDisp12MaxDiff(1);
_sgbm->compute(leftData.imageColor,rightData.imageColor,_disparity_map);
cv::Mat disparity_map_masked(_disparity_map.rows,_disparity_map.cols,_disparity_map.type());
disparity_map_masked.setTo(NO_DISP);
_disparity_map.copyTo(disparity_map_masked, leftMaskMap);
ExportPointCloud(pairName + "_opencv" + ".ply", leftImage, disparity_map_masked, Q, subpixelSteps);

I set subpixelSteps to 16. Then I found that the poincloud generated by ExportPointCloud is wrong. I output disparity_map_masked of opencv sgbm and leftDisparityMap of openmvs sgm to text. And
I found that the two diaparity values are different. The disparity_map_masked is positive value and the leftDisparityMap is negative value. disparity_map_masked.txt leftDisparityMap.txt How should I use opencv sgbm correctly in openmvs? Look forward to your reply. Thank you!

cdcseacave commented 4 years ago

sry, I do not know what OpenCV does to compute the SGBM disparity, or how it stores it; check its documentation and then have a look at how I convert my disparity map to depth map SemiGlobalMatcher::ExportPointCloud()

KeoChi commented 4 years ago

According to my understanding, the left image matches the right image, the value of disparity map should be positive. Why does the value of leftDisparityMap is negative? I found that the disparity value of the two(opencv and openmvs) is exhaustive.

Depth depth(Image::Disparity2Depth(Q, ImageRef(c , r), (float)-disparity / subpixelSteps, u));

Depth value of both is exhaustive. But the poincloud generated by ExportPointCloud is still wrong.