search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://mediapipe.dev
Apache License 2.0
26.77k stars 5.09k forks source link

Head shake occurs with blendshapes #5090

Open wang-xiaochong opened 7 months ago

wang-xiaochong commented 7 months ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Windows 11

MediaPipe Tasks SDK version

0.10.9

Task name (e.g. Image classification, Gesture recognition etc.)

Face landmarker detection

Programming Language and version (e.g. C++, Python, Java)

Python

Describe the actual behavior

The 52 points given when the head is still are jitter

Describe the expected behaviour

No jitter

Standalone code/steps you may have used to try to get what you need

Use the code link: 
https://developers.google.com/mediapipe/solutions/vision/face_landmarker/python#live-stream

Other info / Complete Logs

No response

kuaashish commented 7 months ago

Hi @xiaoshuai-tw,

Thank you for bringing this matter to our attention. Could you please provide any relevant reference images or videos that would assist us more effectively? This information will greatly contribute to our thorough understanding of the issue and will facilitate internal discussions with our team to improve overall performance.

Thank you

wang-xiaochong commented 7 months ago

https://github.com/google/mediapipe/assets/82313318/74916734-8b30-4979-93a7-45aca0080d0c

This is the face_blendshapes taken from the result_callback function in the linked python code above and passed into the UE4 3D digital Man for expression clone display video.There is shaking of the brow and the position of the face and jaw.