search

korejan / ALVR

Stream VR games from your PC to your headset via Wi-Fi
https://alvr-org.github.io/
MIT License
186 stars 27 forks source link

Are there some ports for Eye-Tracking ? #81

Closed Hope10086 closed 1 year ago

Hope10086 commented 1 year ago

Our team is planning to use Pico neo3 Pro Eye or HTC VIVE Pro Eye to collect some user eye movement data while playing. I would like to know if ALVR or ALXR has reserved ports for VR headsets that can provide eye movement tracking? Or can you describe how you combine the OpenXR SDK with the SDK provided by Pico when developing the client? Have you ever used Unity?

korejan commented 1 year ago

@Hope10086 Hello, there is a branch for facial & eye tracking support here (and builds of these are here) however the github version only supports facial/eye tracking OpenXR extensions of Quest Pro, these are vendor specific.

I'm currently working on adding support for more facial & eye extensions, in my local clone. I have added support for XR_HTC_facial_tracking and XR_EXT_eye_gaze_interaction in progress.

XR_EXT_eye_gaze_interaction is a multi-vendor extension for eye tracking and is supported by Pico's OpenXR runtime in newer PUI versions but something to note about this extension, as the name implies, it only outputs a single eye gaze pose as it is intended to be used as another method of input. I don't know if this suits your needs.

Pico's OpenXR runtime does have a vendor specific eye tracking extension however it is unpublished and has no documentation, in older versions of their OpenXR SDK there was a header for some kind of pico eye-tracking API but this has been deprecated & removed in the latest version. I do not know if the current pico extension is the same as I saw in the older Pico OpenXR extensions, it's just something that I will have to find someone to test.

Anyway in the facial & eye tracking branch of alxr, it creates a TCP server that applications like VRCFT can connect to for recieving eye & facial tracking but as I said I'm currently adding support for more extension so in my local version i'm changing the packet format.

Hope10086 commented 1 year ago

Thank you for replying! Maybe we choose Quest Pro for our work . I want to know when eye tracking extensions of Quest Pro has been added to ALXR（roughly）. I pulled your github clone last November . And what are the key variables of the eye tracking module in the client module that might help me quickly understand this part of the code?