search

TadasBaltrusaitis / OpenFace

OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Other
6.86k stars 1.84k forks source link

Gaze with respect to head pose #769

Open Rohit-Varma1 opened 5 years ago

Rohit-Varma1 commented 5 years ago

Hi Tadas,

I was looking into the gaze angles ( x values ) and they go from + to - as I change my head pose from left to right, irrespective of where my eyes are looking at. meaning If i move my head to right...then value for gaze X angles are always negative irrespective of whether my eyes are looking at camera or away. Is there a way to figure out gaze with respect to head pose ? thank you

TadasBaltrusaitis commented 5 years ago

Gaze angles reported by OpenFace are with respect to camera. To get gaze angle with respect to head pose you can just multiply the gaze vector with the inverse matrix describing the head pose.

However, tracking gaze under head pose motion is a difficult problem and there will be a bias in error when the head is moving, so some error in the results is expected. OpenFace is more accurate in terms of relative gaze rather than absolute gaze.

Rohit-Varma1 commented 5 years ago

So you mean multiple the gaze_direction 0 with cv::Matx33f corrected_rotation = camera_rotation * head_rotation; ?

TadasBaltrusaitis commented 5 years ago

Exactly, you might just need to transpose the head rotation matrix (I don't remember which way exactly it should be)

ghdalsrl326 commented 3 years ago

Hello, may I ask a question? I have a similar problem but cannot solve it by multiplying the head rotation matrix or homogeneous transformation matrix to the gaze vector.

My webcam installed just below the monitor which means both are on the same plane. When I looking at the four corner points of the monitor without head rotation, the OpenFace gave me always positive gaze_y value. I cannot understand why this happens.

TadasBaltrusaitis commented 3 years ago

Hi, there may be a systematic error you are encountering, gaze_y is not as reliable as gaze_x. Is the relative change in gaze angle reasonable when looking at different corners of the screen?

AnaisServais commented 2 years ago

Hi, @rohitvarma1986, what you call camera_rotation, is it gaze_angle_x and gaze_angle_x or gaze_0_x, gaze_0_y, gaze_0_z ? Thanks for helping!