https://github.com/deepinsight/insightface/tree/master/reconstruction/gaze
This paper trained a gaze estimation network based on arbitrary unlabeled face images, and used HeadGAN to generate data with different head poses for multi-view constraint. In addition to the gaze vector, the network also predict eyeball meshes, which the author proven to be useful.
Method
Predict both the eye gaze vector and the dense 3D coordinates
Generate ground-truth eyeball mesh by fitting the labelled dataset(MPII)
Generate the pseudo-ground truth eyeball mesh and gaze vector by fitting the unlabeled data through 3D reconstruction network and landmarks alignment
Use HeadGAN[^1] to generate images with different head poses and apply the multi-view consistency loss.
Highlight
The results are good by using the model provided in InsightFace
Enable to train on both labeled and unlabeled data
detailed analysis the existed dataset
Limitation
In section 4, we shown that pseudo-ground truth can be effectively utilized in gaze estimation. In our experimentation, this usually resulted in weaker labels in the pitch axis, which is possibly due to the limited vertical variation of gaze in the data, or flaws of our pseudo-annotation pipeline
What do the weaker labels mean?
Comments
I am not sure whether HeadGAN can preserve the gaze direction as the head pose changes. The image showed in their webpage does not preserve the gaze direction
[^1]: Headgan: One-shot neural head synthesis and editing
Introduction
https://github.com/deepinsight/insightface/tree/master/reconstruction/gaze This paper trained a gaze estimation network based on arbitrary unlabeled face images, and used HeadGAN to generate data with different head poses for multi-view constraint. In addition to the gaze vector, the network also predict eyeball meshes, which the author proven to be useful.
Method
Highlight
Limitation
What do the weaker labels mean?
Comments
[^1]: Headgan: One-shot neural head synthesis and editing