[Question] How to have matching coordinates between the output of a NN and the published image?

[RemiFabre]

Hi,

I'm running a spatial Yolo (V7), and I'm reading the ROS message with the bbox coordinates. I've not been able to have matching coordinates between the published image and the bbox outputs of Yolo:

My understanding is that the NN only works on a 640x640 pixel area inside the rgb image. That area seems to start at x=300 and y=0 of the original image. I tried adding 300 on x on the bounding box coordinates, but the result is still a bit off:

When using the Python pipeline, the result is almost perfect:

I read your post about FOV here: https://docs.luxonis.com/projects/api/en/latest/tutorials/maximize_fov/

So I thought that I would be able to reduce the size of the rgb image to fit the size used by the NN, however the following parameters dont seem to change anything on the image published in "/oak/rgb/image_raw" (tested with rqt_reconfigure, and also by changing the camera.yaml file and restarting the node):

    rgb:
      i_height: 720
      i_preview_size: 640
      i_width: 1280

What is the intended way to transform the bbox coordinates so that they can be used on the published ROS image?

Best,

[Serafadam]

Hi, example detection overlay will be added in depthai_filters, currently it's based on a simple case where it overlays detections on preview image. You can check it out at https://github.com/luxonis/depthai-ros/pull/247. Full FOV option for detections is currently available by changing aspect ratio, you can do that by setting parameter rgb.i_keep_preview_aspect_ratio: true and after that you'll have to resize bounding boxes to fit the detections.

In near future for spatial detection we'll probably add integration for new rviz plugin.

[Serafadam]

Closing the issue, since spatial_bb filter is available. Please feel free to reopen it if you have further questions/comments.