ofxCv::Calibration::undistort should pass output matrix by reference

[xionluhnis]

I struggled for a few hours trying to understand why the result of undistort with my calibration data was always an image with no size.

In my case, my ofApp::update function was:

void ofApp::update() {
    vidGrabber.update();

    if (vidGrabber.isFrameNew()) {
        if (calib.isReady()) {
            cv::Mat grayImage;
            ofxCv::toCv(vidGrabber.getPixels()).copyTo(grayImage);
            calib.undistort(grayImage, rectImage, interpolation);
        }
    }
}

However, the declaration of ofxCv::Calibration::undistort for image undistortion (with input and output) is:

void undistort(Mat src, Mat dst, int interpolationMode = INTER_NEAREST);

and dst is passed to remap, which does the remapping and outputs the data in dst. Except that here, the Mat was empty to start with, pointing to nothing, so remap created the content and put it in dst, which doesn't update the outside rectImage because it's been passed as a copy (a special copy of something that points to nothing, if it did point to something initialized, it wouldn't have triggered the problem).

One solution is to only pass fully initialized Mat objects to undistort, but we can avoid forcing users to do that by changing the declaration as:

void undistort(Mat src, Mat &dst, int interpolationMode = INTER_NEAREST);

and in that case, even if you don't initialize the content of your Mat, it will still be populated correctly.

Added: also for the point undistortion, related to this, OpenCV treats a missing newCamera as an identity, which means we're in [-1;+1]^2 space. Is that wanted? Changing the undistortPoints calls to

undistortPoints(matSrc, matDst, distortedIntrinsics.getCameraMatrix(), distCoeffs, Mat1d::eye(3, 3), undistortedIntrinsics.getCameraMatrix());

makes it go back to the expected undistorted camera space. But maybe it was intended... though then it should be documented.

[hiroyuki]

Hello xionluhnis,

I am working on my project with this now and just found you don't have to do this here. Mat has data pointer in it. So even undistort parameter has no reference of original object, it's has same pointer address in it. So data itself will be updated.

If you can't, I think you should make sure you allocated the pixels before undistort function.

[xionluhnis]

Thanks hiroyuki, so the problem in my code was that the Mat object was not "allocated" before being passed to undistort.

Is this "allocate before passing output reference" a specified design pattern of the opencv library? If it is not, then this seems to me like a bad pattern to require allocation, or at least it should be documented somewhere.

Note that if you pass the output argument by reference, then even if it's not allocated, everything is fine. Is there any reason not to pass by reference then?

[riccardolardi]

Having the same issue here - my image would just stay black e.g. nothing was being done to the (empty, unallocated) mat I passed into undistort()

[xionluhnis]

@alberto2000 does allocating the Mat object solve your problem? It should.

Even though that should solve the problem, I think it's a design issue so I'll keep it open because I don't see any good reason for not passing the output Mat by reference. Passing by reference avoids the need for pre-allocation.

[riccardolardi]

@xionluhnis yes it does, thanks a lot. I agree, it would be much nicer and robust if you could just pass a reference.

[hiroyuki]

@xionluhnis agree. Thanks your comment.