search

zkmkarlsruhe / ofxTensorFlow2

TensorFlow 2 AI/ML library wrapper for openFrameworks
Other
109 stars 16 forks source link

Do you have plans to implement 3d skeleton detection via BlazePose? #17

Closed stephanschulz closed 2 years ago

stephanschulz commented 2 years ago

I just came across this post, 3D Pose Detection with MediaPipe BlazePose GHUM and TensorFlow.js. Do you have plans to make an example on how to use this library. I think it's TF.js pose-detection API ?

As always thank you for your efforts to make tensor flow accessible to the OF community.

bytosaur commented 2 years ago

hey @stephanschulz,

I dont think that we will expand this addon to work with MediaPipe as it comes with new dependencies. This addon only suffices running the functions given by the C library which is only a subset of the functions in the python library.

MediaPipe has a lot of tf.js models which might be converted. Check out their tfhub section, but be careful as a some applications require some code from the mediapipe library in between two models. E.g. hand tracking requires palm detection + mediapipe lib calls + handpose...

I didnt look too much into MediaPipe, but most models are available in C++. To fully make use of MediaPipe you would want to build the examples in an openFrameworks way (i guess without bazel ?). Then publish that as a standalone addon.

danomatika commented 2 years ago

Yeah, I could imagine something like an addon for MediaPipe models which uses ofxTensorFlow2 as a dependency, but it's not something we will probably get into on our end.

stephanschulz commented 2 years ago

Thanks for the input and advice. Implementing this is currently beyond my skills. But it's a great road map to follow.

danomatika commented 2 years ago

Closing for now.