search

davidsandberg / facenet

Face recognition using Tensorflow
MIT License
13.71k stars 4.8k forks source link

Has anyone tried training after alignment using landmarks? #1066

Open vkasojhaa opened 5 years ago

vkasojhaa commented 5 years ago

I was recently going through ArcFace repo, they have used face alignment using MTCNN but they are also using landmarks for alignment unlike in Facenet. I was wondering if anyone has tried face alignment using the landmarks for training Facenet and seen any improvements in the results. I have noticed that MTCNN is able to align images with high pose variance quite nicely.

RubinXnibu commented 5 years ago

I use the 3D version of MTCNN and I separate out images with a yaw > +/- 65 degrees to their own dataset. Getting rid of those has boosted my accuracy on the remaining images. Now I am working on training a face recognition CNN for face profiles for that high-yaw dataset. Training your own CNN from scratch is so slow !!! My idea is to have the two specialist CNN's in parallel and depending on the yaw, push the incoming image through one or the other.

nyck33 commented 5 years ago

@RubinXnibu

Which version of MTCNN outputs pose yaw information?

RubinXnibu commented 4 years ago

Some of the times I am using Adrian Bulat's MTCNN (https://github.com/1adrianb/face-alignment), which gives 3D coordinates for the landmarks, and from there I extract the pitch and yaw of the face.

Here is my code to do so, I hope it is helpful: GetPitchYaw.txt