search

HosniMansour / Unity3DFaceTtracking

Unity 3D RealTime Face Tracking using landmarks generated by OpenFace
https://blog.hosni.me/2018/05/pfa-unity3d-realtime-face-tracking.html
MIT License
113 stars 26 forks source link

How to compute the 50 blendshape weights after getting 68 facial landmarks? #1

Closed seanwen86 closed 5 years ago

seanwen86 commented 5 years ago

Hello sir @HosniMansour , This is a great work. I have learnt from your work how to control the head and the eye of an avatar through webcam. But after getting the 68 facial landmarks, how to compute the blendshape weights in order to control expressions?
I am new to this field. Looking forward to your reply. Thanks in advance.

Best regards, Sean

HosniMansour commented 5 years ago

@seanwen86 Thanks, you can refer to this https://github.com/edk208/mixamoFaceTransfer to understand more how to control expressions (check the Matlab files over there)

seanwen86 commented 5 years ago

Thank you, sir @HosniMansour . I will look into it. Do you have any other materials to learn how to control expressions of an avatar through webcam? I have tried to look for that, but nothing particular, especially for implementation, has been found.

HosniMansour commented 5 years ago

I didn't find a lot of materials to do that in unity but guess you can do that with OpenCV or DLIB, I did write a blog explaining the project and how to use the 68 facial landmarks but it got deleted by accident :( I will try to rewrite it soon

seanwen86 commented 5 years ago

@HosniMansour too bad. thank you anyway for your answers.