how to support yolo model for object detect?

[bugmany]

i 've trained yolo model for object detect, and i want to integrating the model into mediapipe, whether it support or not, if support integrating, how to do, could you give me advice, thanks a lot.

[gustavorps]

+1

[ibrandiay]

@bugmany do you have an answer in your request ? me too i am interest in this question.

[jiuqiant]

@sgowroji, please assign this to the owner of object detection.

[sgowroji]

Hi @bugmany, Which platform are you referring?

[bugmany]

android and iOS.

[ahmadyan]

We do not support Yolo, as it is quite out-dated, instead we recommend to use our SSD MobileNet object detection in Mediapipe.

Typically, if you have a tensorflow model you'll need to convert it to mobile version with tflite_converter. Then use our inference calculator to run inference on device, our tflite inference calculators ensures that the model runs fine on different devices with different GPU backend such as Vulkan, OpenGL, cpu, etc. However, Yolo pre-dates all of this, and comes with its own darknet for inference, and the more recent quantized adoptions like xnornet are not supported. So if you insist on adding this to Mediapipe, you'll need to write a C++ wrapper for Yolo and then directly call that code in Mediapipe, then you have to make sure models runs fine on different gpu backend, etc.

[sgowroji]

Hi @bugmany, Could you please have a look at the comment.

[bugmany]

ok, thanks, then how to "you'll need to write a C++ wrapper for Yolo and then directly call that code in Mediapipe", could you provide demo for information?

[ahmadyan]

You can take a look at Mediapipe Object Detection Demo, in particular the ssd_anchors_calculator.cc

[ibrandiay]

@ahmadyan Thank you for your answers . So if I understood correctly, a yolo model converted to tflite should be able to run normally without needing a C++ wrapper?

[ahmadyan]

If you use tflite you can use our c++ mediapipe wrapper around Yolo for inference. You'll still need to write code to decode the yolo's output and convert it to 2D bounding boxes.

[ibrandiay]

@ahmadyan Basically it was a yolo, but after converting to tflite, the output is fine in 2D. So with these conditions I imagine that it would be enough to run on mediapipe without implementing a c ++ wrapper. Tell me if I'm talking nonsense

[bugmany]

hi @ahmadyan, I'll try it, thanks a lot.

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please reopen if you'd like to work on this further.

[HungDuong1998]

How are the results so far @bugmany @ibrandiay ? I am too interested in the solution

[ibrandiay]

@HungDuong1998 I finally gave up my yolo model to adapt a mobilenet ssd, I have no choice ... The power of google is essential. Yolo is better but he does not have the right support for mobile devices

[HungDuong1998]

What did you do? Have you tried converting to yolo tflite model?

[ibrandiay]

@HungDuong1998 yes

[ibrandiay]

my model is a yolo converted to a tflite but on mediapip it doesn't work

[HungDuong1998]

But not successful still? The yolo tflite is not compatible for google pipeline?

[HungDuong1998]

Do you have any git repo/blog on this? Why it doesn't work? Is it because of the image size input or what?

[ibrandiay]

this is because of the output of the yolo model, you have to write a c ++ wrapper to decode the yolo output. What I don't want to do, in addition you will have problems when you have to run it on GPU. To put it simply, yolo does not have real support for mobiles, unlike google

[ibrandiay]

If you're hot, you and I could consider writing a different c ++ wrapper to adapt yolo v4, yolov5 to support mediapipe. To help the community

[HungDuong1998]

C++ wrapper is easier or modifying the tflite converter tool you used is easier? I am newb to all of this really, would love to help but I don't think I can do much

[ibrandiay]

the yolo to tflite converter tool fits the yolo architecture well, so there is no problem with it. on the other hand, the output must be decoded to be able to insert it on mediapipe. This will be exactly the role of the C ++ wrapper, so it is essential if we want to use the yolo on mediapipe.

[HungDuong1998]

What converter tool did you use? Can you share the reference so I can take a look? I heard some controversy about yolov5 being not official and worse in performance compared to its original v4 and v4 scaled in darknet version

[ibrandiay]

@HungDuong1998 To convert yolo to tflite, use this: https://github.com/hunglc007/tensorflow-yolov4-tflite

Addressing the problem with yolov5, Glenn has done a tremendous job advancing the domain and helping the community. We owe him a lot of respect. Except that the name of yolov5 is not the most suitable since it did not really follow the logic of yolo (research paper, new architechture, ....). The best name should be something like yolo_analytics

[bharath5673]

https://www.linkedin.com/posts/bharaths5673_yolo-mediapipe-cpu-activity-6954752592426004480-dHlP?utm_source=linkedin_share&utm_medium=member_desktop_web

[Gaozhongpai]

https://www.linkedin.com/posts/bharaths5673_yolo-mediapipe-cpu-activity-6954752592426004480-dHlP?utm_source=linkedin_share&utm_medium=member_desktop_web

Hi, can you share your project?