search

pupil-labs / hmd-eyes

Building blocks for eye tracking in AR and VR.
GNU Lesser General Public License v3.0
153 stars 64 forks source link

Support for OpenXR eye_gaze_interaction extension on Linux and Windows? #112

Open kleinerm opened 2 years ago

kleinerm commented 2 years ago

I'm somebody who writes open-source research software for neuroscience and related areas, used by thousands of labs. Atm. I'm trying to find out what hardware options exist for eye tracking with the OpenXR api on VR/AR HMD's, ideally under Linux. A friend of mine uses a regular pupil labs eye tracker under Linux and is quite happy so far, hence the question:

Is there already open-source support available for the XR_EXT_eye_gaze_interaction for your HMD eye tracker extension on Linux, e.g., for use with Monado or SteamVR?

Thanks, -mario

papr commented 2 years ago

Hi Mario, Pupil Labs does not support this API officially. It is possible that community members have worked on this but I am not aware of any concrete examples. The Unity plugin just implements the Pupil Core Network API.. All you need is zmq and msgpack support.

kleinerm commented 2 years ago

Thanks for the quick reply. If I can pose a feature request to you for the future, I'd propose support for that extension. Given that OpenXR is supposed to be the "one to rule them all" cross-platform, cross-vendor, open specification for VR/AR/XR, and does provide a standardized extension for eye tracking, i think it could be quite beneficial. While i don't use Unity, our software (psychtoolbox.org) is custom made, I think Unity does have a plugin to support use of OpenXR and possibly its extensions for VR.

Is the pupil labs VR eye tracker basically treated like any other pupil labs tracker from the software side? I helped somebody to use the zmq interface in the past, and afaik multiple of our users are happily using your non-VR products. Could one use a desktop tracker as "fake"/"surrogate" VR HMD eyetracker for the purpose of software development?

Unfortunately, as a one-man OSS contracting shop, we don't have the money to get a suitable VR HMD or Pupil labs device to do the work ourself atm., unless somebody would contract us to pay for material and time, but at least from the neuroscience research community there could be quite a bit of demand for this in the future, especially once our OpenXR + eye tracking support is ready.

Thanks and have a nice evening.

papr commented 2 years ago

Is the pupil labs VR eye tracker basically treated like any other pupil labs tracker from the software side? [...] Could one use a desktop tracker as "fake"/"surrogate" VR HMD eyetracker for the purpose of software development?

Yes to both!