search

shaoguangcheng / faceFrontalization

Align a face in profile to front view
56 stars 37 forks source link

Need help! #4

Open letrangnhienanh opened 8 years ago

letrangnhienanh commented 8 years ago

Hello, I am trying to test with other images but got fail. Can you please help show me the way running with other images?

Thank for your assistance!

shaoguangcheng commented 7 years ago

@letrangnhienanh ,

Hi! I implemented this algorithm according to paper "Effective Face Frontalization in Unconstrained Images" and I used 49 facial feature points (detected by SDM detector which suggested in the paper) which were included in 'data/facialFeaturePoints.yml'. The facial feature points were used to calculate the parameters of the camera.

So, for other images, new feature points are necessary

orgicus commented 7 years ago

@shaoguangcheng

Hi,

Thank you for sharing the implementation! Great to able to simple get the repo and try it out straight away.

I'd like to change the demo to use a webcam feed and I'm getting a bit stuck. I first tried using'd dlib's facial landmark detector and noticed it the number of feature points doesn't match. Reading through the paper and your comment above I've tried using this SDM implementation, but the number of feature points doesn't match.

SDM returns 68 points, yet facialFeaturePoints.yml uses 49 points. Even if I remove the first 17 points of the face outline (jaw line) that would would still be 51 points. Which points should I use ideally ? (How was facialFeaturePoints.yml saved ?)

Other than the points are there any other changes necessary to run the code using a live webcam feed ?

orgicus commented 7 years ago

Hi @shaoguangcheng

I've tried removing the inner corners of the lips (landmark indices 60 and 64) but the result is still very distorted.

Can you please advise on using face frontalization on a webcam stream (e.g. how should the data (face frame and landmarks) be prepared) ?

Thank you, George

sweatdream commented 7 years ago

@orgicus May I know that did you solve the problem? I also want to change the number of the points. Looking forward to your reply. tks

orgicus commented 7 years ago

@sweatdream I did not manage to solve the problem. I've received some good advice on passing the correct 49 points: index 17 to 68 excluding 60 and 64(inner corners of the lips). Unfortunately this change alone did not solve the problem.

I have even tried to resize/crop the webcam frame to match the dimensions of the sample image, but had no joy.

It's unclear if aside the input image dimensions/face ROI/49 landmarks there is something else that needs to be changed (e.g. camera matrix, 3d model, etc.)? I wonder if @shaoguangcheng has any insights to share.