search

TimoBolkart / TF_FLAME

Tensorflow framework for the FLAME 3D head model. The code demonstrates how to sample 3D heads from the model, fit the model to 2D or 3D keypoints, and how to generate textured head meshes from Images.
http://flame.is.tue.mpg.de/
435 stars 78 forks source link

About extract 3D landmarks from VOCA #47

Open kuanxin opened 3 years ago

kuanxin commented 3 years ago

How to extract the 3d feature points of the obj mesh output by VOCA...>< Many thanks

TimoBolkart commented 3 years ago

May I ask what you intend to do with the landmarks extracted from a VOCA mesh? VOCA outputs meshes in FLAME topology which provides a much denser correspondence than just 51 / 68 landmarks.

However, if you are still interested in the landmarks, just load the FLAME landmark embedding, the FLAME mesh output from VOCA, and call the function mesh_points_by_barycentric_coordinates in here. This returns you the 3D landmarks in that mesh.

kuanxin commented 3 years ago

@TimoBolkart I try to change the 68/51 landmarks of the obj face model in unity3d to achieve the animation effect that the face model is talking. I am not sure is the correct way...

Thanks for your reply! It's very helpful.

kuanxin commented 3 years ago

@TimoBolkart For example, I have a 3D face model generated by DECA, how to use VOCA to make this face model lip-sync in Blender or unity3d?

Thank you

airfri commented 2 years ago

@TimoBolkart
May I ask how to calculate the two formal parameters mesh_vertsandmesh_facesof the function mesh_points_by_barycentric_coordinates from VOCA flame mesh output？

Hello-hubKing commented 4 months ago

@TimoBolkart For example, I have a 3D face model generated by DECA, how to use VOCA to make this face model lip-sync in Blender or unity3d?

Thank you I have same issue, do you resolve it?