Preview window full FOV instead of being cropped

[vladmosoiu98]

Hi!

I was wondering if there are any plans on making it possible for the preview window to use the full FOV of the RGB frame, instead of cropping the image when displaying it on the "/overlay" topic.

I worked with the spatial_bb example and I tried to make this possible by changing the "i_preview_size" but then the detected bounding boxes would be shifted. Any other ways I tried were conflicting either with the NN 300x300 limit or would zoom in even more.

I am asking this, since in the Depthai SDK, that is possible (the way it works in the depthai_demo.py) and it would be nice to have it with ROS as well.

Thanks and hope this package will continue growing!

[Serafadam]

Hi, thanks for the notification, indeed it seems that overlay isn't properly resized before publishing, although if you try publishing Camera source of information from oak/rgb/image_raw topic in Rviz, then you should see that the bounding boxes are displayed properly. We'll add the fix to the overlay + default Rviz config file in near future.

[vladmosoiu98]

Thank you for your response @Serafadam I will try out your suggestion! I will close this issue for now.

[martinlucan]

@Serafadam I also try to make use of spatial_bb.launch. I would like to visualize bounding boxes in full FOV image preview. rgb.i_keep_preview_aspect_ratio: true crops the image and rgb.i_keep_preview_aspect_ratio: false keeps the FOV, but squeezes the image, what I believe is expected. Refering to your previous reply, I do not see any bounding boxes in oak/rgb/image_raw . I tried to set preview_width and preview_height, which adjusts the preview size, but messes up the bboxes. Any opinion on how to solve this out?

[Serafadam]

Hi, spatial_bb node has a desqueeze parameter which will output the image in correct size when i_keep_preview_aspect_ratio is set to false, this will provide correct final image (overlay topic)

[martinlucan]

thank you @Serafadam, unfortunately, a desqueeze param seems to have no effect, the Image on /spatial_bb_node/overlay topic looks completely the same regardless of the setting.

[Serafadam]

@martinlucan You are right, sorry for the confusion, the desqueeze parameter currently is responsible only for resizing the bounding boxes to right FOV, the preview is just copied and overlaid. If you want to just see the bounding boxes, you can display rgb image as Camera and after adding spatial_bbs MarkerArray message they should be visible in camera view (as per screenshots). Alternatively, you could modify spatial_bb.cpp to desqueeze the previewMat itself, following snipped placed just before this line should work.

    // desqueeze previewMat
    if(desqueeze) {
        cv::Mat desqueezedMat = cv::Mat::zeros(info->height, info->width, previewMat.type());
        cv::resize(previewMat, desqueezedMat, desqueezedMat.size(), 0, 0, cv::INTER_LINEAR);
        previewMat = desqueezedMat;
    }