search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.83k stars 5.18k forks source link

BlazePose Full Body #1066

Closed ljmkimqx closed 3 years ago

ljmkimqx commented 4 years ago

Why does Mediapipe only support the upper body and not the full body? When does Mediapipe support the full body? I know the full body function of ML kit, but I prefer mediapipe to support the full body（I think Mediapipe is very practical and convenient ）

mgyong commented 4 years ago

@ljmkimqx I would encourage you to use MLKit if you need the full body for now. Releasing something in MediaPipe might be something we will consider. Thks

vpenades commented 4 years ago

@mgyong in our case we're glad to support MLKit on Android and iOS, but our project needs to be available on windows and linux too, for which MLKit is not available. Right now we're considering alternatives.

mgyong commented 4 years ago

Windows support is experimental for us. Could you share with me what your project is about?

vpenades commented 4 years ago

We've been able to successfully run the upper body model in windows, and it just runs fine.

Our project is for a potential client that wants a virtual coach app. For most use cases, 10fps on mobile is fine, but there's a few specialised cases that might require 30fps, for which we think a PC is better suited (for example, boxing)

Obviosly, in PC there's depth cameras and other alternatives. But ideally, we would like to support a single codebase, instead of having to support different models on different platforms. Also PC depth cameras are very demanding and not always available, so if we can lower the minimum requirements of the hardware requirements by using BlazePose, that would be a plus.

jiuqiant commented 4 years ago

@vpenades, I assume that you already solved this Windows issue and ran the upper body model on Windows. Can you post your fix of "mediapipe/mediapipe/framework/packet.h" to benefit the other Windows users? Thanks.

vpenades commented 4 years ago

@jiuqiant I'm afraid I cannot help in that issue; I am developing in C# and using Emgu.TF for TensorFlow Lite inferences. It took me a while to understand how to use the model, I also had a lot of good luck; in the past I was able to run the BlazeFace model in Emgu.TF, but I was not able to run the hand landmarks model.

Certainly, I would have liked to use MediaPipe directly, but since there's no c# bindings for it, it's no use for me.

On Android and iOs, I hope to use the MLkit component through Xamarin c# bindings when they're available.

Regarding your windows issue; if it's compiling on gcc but failing with MS BuildTools... I would file an issue to Microsoft. Maybe it's a bug on their side... or maybe it's yet another STL inconsistency; Microsoft is notorious when implementing STL in its own way, sometimes with good reasons.

PINTO0309 commented 4 years ago

Tensorflow.js + BlazePose Full Body + Browser + WebGL

https://www.youtube.com/watch?v=UaWX-nz5gnI

ezgif com-optimize (1)

CoreML, TFLite, TF-TRT, OpenVINO, TF.js Models

https://github.com/PINTO0309/PINTO_model_zoo https://github.com/terryky/tfjs_webgl_app

sgowroji commented 3 years ago

We are closing this feature request as this is fixed and available in our MediaPipe solutions.

google-ml-butler[bot] commented 3 years ago

Are you satisfied with the resolution of your issue? Yes No