google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
26.97k stars 5.1k forks source link

model output in embedded device with C? #4234

Closed RTL8710 closed 1 year ago

RTL8710 commented 1 year ago

How to efficiently and succinctly convert the output of the model into the structure of the C language by porting the mediapipe model (object_dection) to the arm embedded system? image

ayushgdev commented 1 year ago

@RTL8710 Would you please elaborate your query with complete details like the use case you are addressing, the embedded system you are targetting, and what do you mean by "structure of C language"?

RTL8710 commented 1 year ago

Background: I would like to port the Mediapipe face dection model (face_detection_short_range.tflite)to embedded device(this part has been completed), question:use Python for model post-processing as shown in the screenshot? image How do I need to parse Python like structures detection in embedded (C language)?

ayushgdev commented 1 year ago

@RTL8710 MediaPipe uses PyBind11 internally for C++ integration in Python. However, in your use case, you need to parse output by python in C language. This can be done by embedding the python interpreter in C. Please check the PyBind11 documentation here for such use case.

RTL8710 commented 1 year ago

Thank you very much for your answer!

Introducing the entire Mediapipe project requires the use of PyBind11, but using PyBind11 is too large for embedded systems, so it is not suitable. My embedded system TPU can already parse the output of the model, but how can the post-processing of the model output concisely parse the detection data structure (similar to the screenshot section)? image

RTL8710 commented 1 year ago

I looked at the source code TfLiteTensorsToDetectionsCalculator::ProcessCPU in the last few days and found the answer I needed.

ayushgdev commented 1 year ago

@RTL8710 Would you please share on the solution that worked for you so that the community can also benefit for similar future requirements?

RTL8710 commented 1 year ago

DecodeBoxes is model output decode,C language overwrite it。

google-ml-butler[bot] commented 1 year ago

Are you satisfied with the resolution of your issue? Yes No