search

roboticslab-uc3m / vision

Vision processing
https://robots.uc3m.es/vision/
11 stars 7 forks source link

Adopt yarp::cv:: utilities for cv::Mat <-> yarp::sig::Image conversions #95

Closed PeterBowman closed 4 years ago

PeterBowman commented 5 years ago

The YARP_cv component was introduced in YARP v3.2.x. Available helpers: yarp::cv::toCvMat and yarp::cv::fromCvMat (ref).

It would be nice to avoid unnecessary IplImage wrappers and explicit BGR<->RGB conversions. See https://github.com/roboticslab-uc3m/vision/commit/d2fdc97fbe98d44e61d4528f655aacfa95a18ea4 (not ready). Note that getIplImage() and wrapIplImage(), as well as the yarp/sig/IplImage.h header, have been deprecated.

PeterBowman commented 4 years ago

Also keep in mind https://github.com/robotology/yarp/issues/2161#issuecomment-682094974: yarp::cv:: methods do not perform a clone of the original object, therefore care must be taken to not exceed the lifetime of the object that handles memory allocation.

PeterBowman commented 4 years ago

Currently using (same in HaarDetector.cpp):

https://github.com/roboticslab-uc3m/vision/blob/7497b4b07688aafa101889f5285abddc1c16c427/libraries/YarpPlugins/ColorRegionDetector/ColorRegionDetector.cpp#L69-L71

Proposal, yet to be tested:

yarp::sig::ImageOf<yarp::sig::PixelBgr> inYarpImgBgr;
inYarpImgBgr.copy(inYarpImg); // copy and align pixels in memory using the BGR layout
cv::Mat inCvMat = yarp::cv::toCvMat(inYarpImgBgr); // no copies nor conversions involved!
PeterBowman commented 4 years ago

For future reference, it turns out the preferred way to duplicate and assign a pixel layout in memory, thus possibly choosing a different color space than the original image, is achieved by calling Image::copy():

using T1 = /* source pixel type, e.g. yarp::sig::PixelRgb */;
using T2 = /* target pixel type, e.g. yarp::sig::PixelBgr */;

yarp::sig::ImageOf<T1> img1 = makeImage()
yarp::sig::ImageOf<T2> img2;
img2.copy(img1); // copies img1 into img2, adjusts layour accordingly if T1 != T2

Both YARP and OpenCV work with images at the memory level, but YARP adds a typed layer which helps abstracting the pixel layout. In the above example, for pixel (x,y): img1(x,y).r == img2(x,y).r (this is correct since we are dealing with an exact copy of the image). If we did img2.setExternal(img1) instead, the result would lean closer to the way one would think of cv::Mats: img1(x,y).r == img2(x,y).b.

Edit: per Image::setExternal()'s docs: Make sure to that pixel type and padding quantum are synchronized (you can set these in the FlexImage class). Therefore, img2 must be a FlexImage in the non-copy scenario.