search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://mediapipe.dev
Apache License 2.0
26.77k stars 5.09k forks source link

Possibility of calculating facial blendshapes from facial landmarks #5587

Open neutronengas opened 2 weeks ago

neutronengas commented 2 weeks ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Linux Ubuntu 22.04

MediaPipe version

0.10.14

Bazel version

No response

Solution

Holistic / Face

Programming Language and version

Python

Describe the actual behavior

We employed the legacy solution on a large video dataset. We refrained from utilizing the new solutions due to accuracy problems (see our other issue #5569). We thus only possess the facial landmarkers as calculated by the legacy model

Describe the expected behaviour

We would also like to obtain the facial blendshape scores.

Standalone code/steps you may have used to try to get what you need

Is there any way to obtain the blendshape scores post-hoc from the facial landmarkers we calculated with the legacy model?

Other info / Complete Logs

No response

kuaashish commented 2 weeks ago

Hi @neutronengas,

Thank you for raising this request again. As previously noted, the current face landmarker does not return blendshape scores. However, we will share this request with our team and discuss it further. Please note that we cannot commit to its availability or guarantee implementation at this time.