Q: Eye-Tracking Applications

[lorenzleutgeb]

Hi!

I am considering buying an Intel RealSense D435 with the goal of using it as (a) 3D scanner, (b) head tracker, (c) eye tracker. I already own a Tobii 4C eye tracker, but Tobii's support for Linux, or in general more open use of their hardware, is very bad so here I am.

This issue is purely about eye tracking. I found GazeSense (commercial product) that seems to do eye tracking via Intel RealSense D415, claiming 2-5 degrees of accuracy according to their specs.

Did anyone else implement eye tracking with an accuracy of <3 degrees with Intel RealSense (also with other devices, not just the D435)? Please point me at some code!

Somewhat related:

• 7730 (mentions eye tracking code sample for SR300)

• 8095 (talks about face tracking, which is related to eye tracking and head tracking more generally)

[MartyG-RealSense]

Hi @lorenzleutgeb Eyeware GazeSense is the main RealSense-compatible commercial solution available for eye / gaze tracking. It has also been highlighted in official Intel RealSense articles.

https://www.intelrealsense.com/types-of-tracking-overview/

https://www.intelrealsense.com/software-for-intel-realsense/#eye-tracking

In regard to open-source solutions: if you are able to use Python then the PyGaze toolbox could provide a pre-made means of achieving eye tracking.

http://www.pygaze.org/

https://github.com/esdalmaijer/PyGaze

You could also use the dlib face detection library, like the tutorial for Python, OpenCV and dlib in the link below.

https://towardsdatascience.com/real-time-eye-tracking-using-opencv-and-dlib-b504ca724ac6

I recommend googling for opencv eye gaze to research other ways of implementing eye tracking with OpenCV.

The RealSense SDK has a compatibility wrapper for dlib that includes a face landmark detection example:

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/dlib/

[lorenzleutgeb]

OK, thanks for those pointers! I also found this similar discussion in Intel Community which also mentions GazeSense and the idea of integrating with PyGaze.

To be honest, I was hoping that there are more people experimenting with eye tracking on Intel RealSense, but that's okay :)

[MartyG-RealSense]

Hi @lorenzleutgeb Do you require further assistance with this case, please? Thanks!

[lorenzleutgeb]

No further assistance is required. I did not close the issue, hoping that other people might bring up more examples/applications/codebases, but that probably won't happen.

[MartyG-RealSense]

Thanks very much @lorenzleutgeb for the update!