search

vladmandic / human

Human: AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition, Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction, Gaze Tracking, Gesture Recognition
https://vladmandic.github.io/human/demo/index.html
MIT License
2.35k stars 323 forks source link

gaze detection logic & comparison #394

Closed arvindsaraf closed 1 year ago

arvindsaraf commented 1 year ago

Issue Description

Hi - I was trying to understand the Gaze detection logic in https://github.com/vladmandic/human/blob/main/src/face/angles.ts I'm assuming it uses the eye angles along with face angles? Is there a pointer to the logic being used? Also, any papers on how does it compare with a purely CNN based Gaze detection models (ie those that output the gaze models as NN outputs instead of calculation from the face features) such as L2CS Net?

Thanks

vladmandic commented 1 year ago

gaze detection is not an ml model, its simple math based on iris position tracking combined eye box that is calculated from overall face detection. for example, left means ratio of iris distance to left corner vs right corner of the eye. since corners get contracted as face rotates, there is no need to add gaze direction on top of face angle.

how does it compare to pure gaze detection models? poorly. but lightest gaze detection gan i've seen is about 100x in size and 10x in computational intensity and gaze detection is secondary function in human, not a primary one that would justify such heavy-handed approach.

btw, since this is a question - in the future can you open a discussion topic instead of an issue.