How to convert face landmarks to canonical space

[domisjustanumber]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

No

OS Platform and Distribution

web

MediaPipe Tasks SDK version

0.10.3

Task name (e.g. Image classification, Gesture recognition etc.)

Face landmark detection

Programming Language and version (e.g. C++, Python, Java)

node js

Describe the actual behavior

I am able to get the facial landmark data in the 0-1 range and scale it to render as an overlay to the original video, and it matches up perfectly. This is great.

I am also able to build a 3D model in the same coordinate space as the canonical face model and use the transformation matrix to render my model to a face in the original video. Also amazing!

What I'm missing is a way to translate the facial landmark mesh into the same coordinate space as the canonical model so I can render interactions between the face landmark mesh and my translated 3D object. They're off by a factor of 20 or so right now.

Describe the expected behaviour

It looks like there was a face_geometry model is of use here, but I'm a little lost on how to use it purely within node.

Any thoughts on how to do this translation?

Standalone code/steps you may have used to try to get what you need

Mostly a math question I think? :)

Other info / Complete Logs

No response

[kuaashish]

@domisjustanumber,

This can be used with the result from FaceLandmarker to draw the face geometry in JavaScript :

https://github.com/google/mediapipe/blob/master/mediapipe/tasks/web/vision/face_landmarker/face_landmarks_connections.ts

Thank you

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No