search

asus4 / tf-lite-unity-sample

TensorFlow Lite Samples on Unity
873 stars 254 forks source link

Determining Face Rotation In FaceMesh Sample #364

Closed cancaglar closed 1 month ago

cancaglar commented 4 months ago

Hello, I am trying to determine if a face is looking at the screen in the FaceMesh example. To do this, I need the rotation of the face. I have tried a bit but couldn't manage it. I looked into how it's done with MediaPipe in Python, but couldn't find much useful information. Can you help me with this?

stale[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stephahn commented 1 month ago

Hello if it is still an issue, I succeed to have rotation around the 3 axes using the Kabsch algorithm with data input being the predicted landmarks, and the canonical face model. That output are the yaw, pitch and roll (I added the scale also in the resolution).

Otherwise, mediapipe has in his solution a transformation matrice as output that should contains the rotations. But I never succeed to use the output matrice and the documentation was not clear.

For speed purpose I recommend to code it in C++ with Eigen library.

good luck