search

TadasBaltrusaitis / OpenFace

OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Other
6.81k stars 1.83k forks source link

Why the 3D landmarks are unsymmetry? #236

Open ywjcrazy opened 7 years ago

ywjcrazy commented 7 years ago

Dear Prof. Baltrusaitis, The 3D landmarks (corrected by the head rotation) of a given frontal face is always unsymmetry, as in the Figure 1 below. I suspect that it may be due to the view angle, so I calculated the amplitude of the left/right lip corners, as displayed in Figure 2. We try the outter-corners of the eyes (because they don't actually move) and plot the amplitude and the resluts displayed in Figure 3. We found this problem in most of the samples. Would you please help me figure it out?

Best regards!

Wen-Jing Yan Wenzhou University, China

Figure1: fat face

Figure2: amplitude of lip corners

Figure3: eye_outer_corners

TadasBaltrusaitis commented 6 years ago

The reason for the asymmetry is possibly because the 3D model does not fully decouple rotation from identity. The 3D shape parameters are constructed in an automated way and are intended to capture identity (face shape) and expression (face shape variations due to smiles etc.). However, they also end up encoding rotation. This leads to 2D data possibly being represented by a 3D face that is rotated when it should not be. The images you attach are they from the same frame? It does look like they are more asymmetrical than they should be. What is the exact script you used to extract the data?