search

david-wb / gaze-estimation

A deep learning based gaze estimation framework implemented with PyTorch
161 stars 31 forks source link

How to get [Yaw, Pitch, Roll] or gaze vector [x, y, z] #1

Closed mohammad-saber closed 4 years ago

mohammad-saber commented 4 years ago

Thank you very much for sharing your code. Is it possible to get the gaze vector? I mean [Yaw, Pitch, Roll] or [x, y, z] values of gaze vector with respect to eyes. Thank you very much.

david-wb commented 4 years ago

Hi,

The model is trained to directly predict pitch/yaw of the gaze direction. During preprocessing the [x, y, z] look vector from unity-eyes is converted to pitch/yaw angles using the util.gaze.vectory_to_pitchyaw() function. There is no roll value.

The formula used is this

theta = asin(-y) phi = atan2(-x, -z)

The MPIIGaze project mentions the same formula here https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/gaze-based-human-computer-interaction/appearance-based-gaze-estimation-in-the-wild/

Hope that helps.

mohammad-saber commented 4 years ago

Thank you very much for your help.

Could you please let me know the direction of XYZ axes. For example, Z axis is upward or downward?

And, I guess that the unit of Yaw, Pitch is Radian. Is it correct?

david-wb commented 4 years ago

For the unity eyes data, I believe z-axis points into the screen, y-axis points up and x-axis points right, which is a bit confusing because it is left-handed coordinate system.

There is also a function util.gaze. pitchyaw_to_vector that does the conversion from from pitch/yaw to [x,y,z] vector. You might also find the notebook notebooks/explore_data.ipynb helpful.