search

ruxailab / web-eye-tracker

MIT License
3 stars 17 forks source link

Improvement: combine the existing model with a headpose estimation using the nose #19

Open KarinePistili opened 2 months ago

KarinePistili commented 2 months ago

We would like to test out the mix of two strategies. Use the linear regression model we have implemented together with a headpose estimation technique to check out if it improves our results

gitone912 commented 2 months ago

hey @KarinePistili , i d like to the head pose estimation implementation along with gaze tracking , while working on different models for the same i tried some 2-3 models and methods combination and one was this so i d like to contribute on this part.

KarinePistili commented 2 months ago

Hello @gitone912 , sure you can contribute to it, how do you plan to do the implementation?

gitone912 commented 2 months ago

the methods i looked upon till now for this are quite interesting ones 1st i checked how v tubers use eye gazing and head pose estimation to beat the latency and stream live so perfectly by moving their pupils cheeks tongue. 2nd i checked upon some libraries in which mediapipe is the most reliable and working till now and i am thinking to implement that and merge that now along with exploring it more.

you can see the workings here of head pose , its jst an example , the integration of eye gaze is left as well as the data and model changes are left.

https://drive.google.com/file/d/1vvSi5AoLCniX3t079B6porT3WnsRGQc0/view?usp=sharing

KarinePistili commented 2 months ago

Huum okay, you are trying an example with tensorflow python correct? Maybe try one with Javascript to see if it can be integrated along with the calibration system to see if it would help out the model.