opencv / opencv

Open Source Computer Vision Library
https://opencv.org
Apache License 2.0
79.06k stars 55.83k forks source link

Stitcher generates panorama smaller than input #17803

Open chacha21 opened 4 years ago

chacha21 commented 4 years ago

OpenCV 4.3.0

I am currently trying the OpenCV Stitcher on very degraded images just to check for the limits of mosaicing. For the algorithm not to hang, I have to set the bundleAdjuster to NoBundleAdjuster,

I also set WaveCorrection to false because it is not a linear travelling

Is it expected behaviour that the pano of two images can be smaller than input ? For two 640x480 images, the resulting pano is 279x257. (with status being OK, not an error like ERR_NEED_MORE_IMGS)

Since the compositionResol is at -1 (ORIG_RESOL), I do not understand how it can happen.

Is there anything else to tune to force the resulting pano to be at least the size of the largest input ?

[edit] The two images are almost identical (the camera did not move yet, only lighting and noise changed a little) I tried with different bundle adjusters, and still the same : the pano is smaller, while it should have matched the two images almost perfectly

chacha21 commented 4 years ago

I have made another attempt with two other images of size 640x480. They are almost identical, but the pano is 743x27. Is it a bug ?

image2

image1

  cv::Mat im1 = cv::imread("image1.png", cv::IMREAD_UNCHANGED);
  if (im1.channels() == 1)
    cv::cvtColor(im1, im1, cv::COLOR_GRAY2RGB);
  else if (im1.channels() == 4)
    cv::cvtColor(im1, im1, cv::COLOR_RGBA2RGB);
  im1.convertTo(im1, CV_8UC3);
  printf("im1 type = %d, size(%d,%d)\r\n", im1.type(), im1.cols, im1.rows); //640x480, 8UC3

  cv::Mat im2 = cv::imread("image2.png", cv::IMREAD_UNCHANGED);
  if (im2.channels() == 1)
    cv::cvtColor(im2, im2, cv::COLOR_GRAY2RGB);
  else if (im2.channels() == 4)
    cv::cvtColor(im2, im2, cv::COLOR_RGBA2RGB);
  im2.convertTo(im2, CV_8UC3);
  printf("im2 type = %d, size(%d,%d)\r\n", im2.type(), im2.cols, im2.rows);//640x480, 8UC3

  cv::ocl::setUseOpenCL(false);
  cv::Ptr<cv::Stitcher> stitcher = cv::Stitcher::create(cv::Stitcher::PANORAMA);
  auto bundleAdjuster = cv::Ptr<cv::detail::BundleAdjusterAffine>(new cv::detail::BundleAdjusterAffine);
  stitcher->setBundleAdjuster(bundleAdjuster);
  std::vector<cv::Mat> imagesPair = {im1, im2};
  cv::Mat nextPano;
  cv::Stitcher::Status status = stitcher->stitch(imagesPair, nextPano);
  printf("status = %d, nextPano size(%d,%d)\r\n", status, nextPano.cols, nextPano.rows);//743x27