search

vardanagarwal / Proctoring-AI

Creating a software for automatic monitoring in online proctoring
MIT License
540 stars 329 forks source link

eye tracking with tensorflow model #53

Closed soans1994 closed 3 years ago

soans1994 commented 3 years ago

hello author,

cheers for the great work. i have some queries regarding your implementation. i read your article on how tensorflow overcomes the dlib landmarks. but while testing there are some false detections. is it because of the dataset used when training?

  1. the tensorflow version of landmarks which you linked with other author works good. but it detects the keypoints even when the face parts are covered. for example when i cover my eyes or nose, there is false detection. how can we overcome this?
vardanagarwal commented 3 years ago

Cannot necessarily point this to the dataset, beacuse this model always return 68 points without any confidence values for them. So maybe if the confidence could be taken into account and provided the dataset has these cases then I guess this problem could be overcomed.

soans1994 commented 3 years ago

Thank you for your reply,

  1. How is this different from openpose or tfpose. When testing those models, the occlusion is considered, if the eyes are covered, it won’t detect those key points . Is it because of the coco dataset which consists of occlusion flag.
  2. can I not set threshold for the heat maps in tensorflow?
soans1994 commented 3 years ago

Cannot necessarily point this to the dataset, beacuse this model always return 68 points without any confidence values for them. So maybe if the confidence could be taken into account and provided the dataset has these cases then I guess this problem could be overcomed.

does this mean that if coco dataset is used with this model, it detects all keypoints? please can you tell me the difference between openpose and your method.