Align RGB with Depth map

[metanav]

I am trying to align depth map to the RGB at the host. I am using spatial_mobilenet.py as an example but I am using an object detection model at the host not on the device. The bounding boxes in the RGB image is properly working but the bounding box on the color depth map is always off.

[Erol444]

Hi @metanav , Perhaps you could run RGB-depth alignment on the device (example here) and then still run inference (AI) on the host side? Instead of consuming host-side resources, you can just run it on RVC for which the alignment algorithm has been optimized. Thoughts? Thanks, Erik

[metanav]

Thanks, @Erol444 for the reply. I am using a custom SSD Mobilenet V2 model which was trained using TF 2.7. I could not convert the model using the model optimizer. I guess the model optimizer has support for SSD mobilenet models only up to TF 2.4. Although I tried with all available SSD API support configs at the OpenVINO repo none of them worked. So I wanted to use the model on the host side. I tried the RGB-depth alignment example with the spatial calculation at host which seems working fine. Unfortunately I can't take advantage of the on-device functionalities.

[Erol444]

Hi @metanav , Regarding the ML issues (TF 2.7...), please create an issue on depthai-ml-training repo (I tried transferring this one but for some reason it doesn't work). Thanks, Erik