search

elucideye / drishti

Real time eye tracking for embedded and mobile devices.
BSD 3-Clause "New" or "Revised" License
392 stars 82 forks source link

API for android call #739

Closed StellaSong12 closed 5 years ago

StellaSong12 commented 5 years ago

Hello, I'm glad for building successfully in android mobile. Then I hope to get the position (x, y) that eye blow stare at, or the direction it move to. Is there any existing API for android jni to call? Or is there any documents and suggestion for me to read?

Really appreciate for your reply!

headupinclouds commented 5 years ago

There is no high level gaze API, but you can monitor the low level 2D models from the eye tracking over time in the SDK callbacks for simple relative gaze measurements here:

https://github.com/elucideye/drishti/blob/b9cadaec0758440f6d4e94c469bc3a48f00a8af5/src/lib/drishti/drishti/ut/test-FaceTracker.cpp#L224-L230

You can register that callback to monitor estimated face + eye models in real-time, and if you find something of interest, you can optionally return a structure requesting a buffer of textures and/or frames at that point in time (grabbing textures is free, but GPU->CPU transfers are slower and should be a once-in-a-while operation). If relative gaze estimates are sufficient for your application, you can estimate them using changes in the pupil or eye center positions wrt a stable reference coordinate system. If you need some absolute gaze estimate, you will probably have to train an application specific model with an image based CNN approach.

StellaSong12 commented 5 years ago

Thank you a lot! It really helps!