custom object detection with yolov5 model

[venkataprasad-git]

Preliminary Checks

• [X] This issue is not a duplicate. Before opening a new issue, please search existing issues.
• [X] This issue is not a question, bug report, or anything other than a feature request directly related to this project.

Proposal

hi zed folks

i tried running my own pytorch model subscribing image and pointcloud also processing into 3d coordinates while running i found it is running very slow then the your inbuilt detection models you have , that to in NVIDIA AGX . so i thought of running your own custom object detector on samples which is running in real time so if possible please give any update on this or include it on the zed wrapper so it will launch with the weights. 2) publish a document for code modification for changing these default parameters 3) try adding semantic segmentation models also

Use-Case

to use custom models which your inbuilt one's can't detect could be helpful for self driving car application and hope it increase the FPS output or sync with the pointcloud and bounding box data

Anything else?

please give any update if there are any other ways i can do it

[Myzhar]

Hi @VenkataPrasadBoson is this issue related to robotics and the ROS wrapper? Are you trying to use the custom detector with ROS?

[venkataprasad-git]

yes this is while using ROS wrapper its output is very less in 10HZ i tried subscribing pointcloud and image for getting 3d bounding box processed with pytorch models and i have also tried with legged robotics darknet package the output was similar i understanding there are too many processes going over it but if i enable inbuilt object detection package everyting seems to work fine

2) yes i tried with all the different packages if you can create an inbuilt process this could help us

[Myzhar]

Hi @VenkataPrasadBoson our AI models are optimized for the used GPU by using Nvidia TensorRT, that's why they are faster. You should do the same if you want to reach the same performance.

2) publish a document for code modification for changing these default parameters

What parameters are you speaking about? Can you explain it better?

3) try adding semantic segmentation models also

There are no ROS topics that allow handling this feature. Any suggestion?

[venkataprasad-git]

hi @Myzhar

sorry for the late replay 1)ok i will try to use Nvidia TensorRT 2) about adding custom object detection including into the zed_wrapper or to do documentation for user to do customization's of complete zed wrapper if possible there may be a lot of chances to get recognition from open source community 3) Im worked with so many simulators they do publish on image msg for publishing

[Myzhar]

OK, so they "simply" publish image topics with segmented objects. This can be a good feature to add.

[venkataprasad-git]

thanks @Myzhar i'm getting huge performance difference with it, thanks for recommendation for using tensorRT

[tugbakara]

You should do the same if you want to reach the same performance.

Hi @Myzhar , I want to achieve this part , I want to use ROS with ZED cam to run object detection of my custom model but where I need to put tensorRT o/p of yolo into zed wrapper?