Closed pperle closed 3 years ago
Hi, @pperle
I think it's due to the different orientation of the axes, origin, and scale of the 3D model. https://github.com/hysts/pytorch_mpiigaze/blob/master/gaze_estimation/gaze_estimator/common/face_model.py#L12-L28
Applying the following patch should fix the problem.
--- a/gaze_estimation/gaze_estimator/common/face_model.py
+++ b/gaze_estimation/gaze_estimator/common/face_model.py
@@ -580,7 +580,9 @@ class FaceModel:
[4.530000, 2.910000, 3.339685]
],
dtype=np.float)
- self.LANDMARKS /= 10.0 # cm to mm
+ self.LANDMARKS /= 100 # cm to m
+ self.LANDMARKS -= self.LANDMARKS[1]
+ self.LANDMARKS *= np.array([1, -1, -1])
dlib_map = [127, 234, 93, 132, 58, 172, 136, 0, 152]
Thank you for your help and the code for the needed 3d model modifications.
Thank you for making the project so modular and easy to expand.
Because I have problems with the dlib face detector in non-ideal lighting conditions I would like to replace it with the face detector of mediapipe. Please see my fork.
For some reason the three axis do not align with the 3d head pose. I suspect there is something wrong with my mediapipe landmarks. The rotation matrix of solvePnP looks fine, but maybe the translation vector is off.
Do you have any idea what might be the problem?
This is the output with mediapipe, with all the axis pointing upwards:![image](https://user-images.githubusercontent.com/17356275/113569364-acf13a00-9612-11eb-8028-aed876d96205.png)
And this is what the landmarks of mediapipe look like:![landmarks](https://raw.githubusercontent.com/google/mediapipe/master/mediapipe/modules/face_geometry/data/canonical_face_model_uv_visualization.png)