Mouth-only feature detection for VR applications

[DrBomb]

MediaPipe Solution (you are using)

mediapipe==0.10.0

Programming language

Python

Are you willing to contribute it

Yes

Describe the feature and the current behaviour/state

Right now, a full face is required to detect mouth features. Covering the face makes the tracked features and points to freak out as the system is looking for a whole face.

Will this change the current API? How?

I assume so. Perhaps it would reuse some parts of FaceLandmark detection but it could have only mouth detection as a separate task.

Who will benefit with this feature?

Any person hoping to use mouth tracking with their VR headset.

Please specify the use cases for this feature

There are some VR headset addons that use an external module to detect mouth features to use in VR applications such as VRChat. Mouth expression detection is already present in Mediapipe, so limiting only to mouth would enable this use case.

Any Other info

Most likely face detection is crucial to the whole landmark detection, I understand if its impossible currently. I would appreciate pointers on where to start in case you'd believe it is possible.

[kuaashish]

@DrBomb,

Thank you for raising this request, We are forwarding request internally based on discussion team will prioritise this work.