Error: (-215:Assertion failed) Rect(0, 0, src.cols, src.rows).contains(cT) in function 'cornerSubPix'

[bartoszptak]

Hi, Thanks for your excellent work! I'm running your code on my stereo+imu rosbag recordings using ROS Humble. In my data, camera frames are published with 4 FPS, resulting in the error when the move between frames is a little higher:

[vio_ros-2] terminate called after throwing an instance of 'cv::Exception'
[vio_ros-2]   what():  OpenCV(4.9.0) /home/host/Softwaere/opencv/modules/imgproc/src/cornersubpix.cpp:99: error: (-215:Assertion failed) Rect(0, 0, src.cols, src.rows).contains(cT) in function 'cornerSubPix'
[vio_ros-2]

I debugged the code and discovered that the issue exists in this place because in the pts2 vector exist points out of image boundaries:

[vio_ros-2] min x: 10.3643
[vio_ros-2] max x: 1216.86
[vio_ros-2] min y: 101.243
[vio_ros-2] max y: 748.67

when the frame shape is 1280x720 (downscaling is set to 1.0).

When I commented on the cv::cornerSubPix function, the VO operated fine. Do you have the best idea how to handle that? Should I just filter out the wrong points, or is the topic much deeper?

[cdebeunne]

Hi, thanks again for your comment. Commenting the line is not a good fix in the first place, as a track outside of the image should not exist. How did it happen?

• Is the initial point already outside the image? Then the issue is about the point extractor
• Have both images the same size? I believe that opencv should raise an exception too, but I don't know exactly
• Has the tracker gone too far? Maybe the size of the search area is too big for your configuration

Try to investigate this, it could be great to have a clear idea of how such a thing happen. Personally, it is the first time I see such an error.