yolo pose estimation

[yassine-kassis]

Hello, Great Work ! is it simple to do the same thing for yolo pose estimation (keypoint detection) ?

[spring98]

hello!

Thank you for your interest in my package.

Actually, I can't give you a definite answer because I haven't done pose estimation, but after a quick investigation, it doesn't seem to be much different internally.

In conclusion, I don't think it's difficult to do the same thing.

thank you

[yassine-kassis]

Thanks ! I'll be intresting to have a discussion with you to learn more a about this package. I tried before tflite_flutter package and it takes more time to run a yolov5s model, why yours is real time?

[spring98]

That's a good question.

In my view, the reason is as follows.

1. To use the package, you must use the camera library.

2. The moment you do cameraController.takePicture(), all the binary data of the photo is transferred from native to Flutter, and it is sent back to native for inference and brought back to Flutter. Of course, the same applies to cameraController.streamListen().

3. Because such a large amount of data is moved redundantly, costs increase and performance deteriorates.

I didn't want to do that, so I accessed the camera in each native, made inferences with that data right away, and drew the view internally. After that, I ported it to Flutter's widgets, UIKitView and AndroidView, and made it usable like a widget in Flutter.

If you have any questions, feel free to ask more.

[yassine-kassis]

I see ! Great job ! But the nms take the most time always, I can't see how it's done in your implementation

[spring98]

Sorry for the late reply!

As a result of measuring the execution time of each function, unfortunately, most of the time was spent in the forward function that passes the yolo model rather than nms.

[This photo was tested with Galaxy S10.]

Additionally, the time complexity of nms is O(n^2), and n is the number of objects detected during that frame, so it will not take tens of milliseconds during normal use. (There may be a slight difference depending on AP performance. ..)

[yassine-kassis]

I like your work! I would be interested to communicate woth you. This is my email : yassinekassis6@gmail.com. Send me a message if you have the time ! Thanks !