search

microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6k stars 2.12k forks source link

There is no way to get eye gaze information and also use head + airtap as primary input method #7220

Closed julenka closed 4 years ago

julenka commented 4 years ago

Describe the bug

Let's say I want to use hand head gaze for input, but still know where the user is looking. I would like to call InputRayUtils.TryGetEyeGaze to get this information, however currently that method will return false if the EyeGazeProvider is not the primary gaze provider for MRTK. I wish there was a way for me to just know where the eyes were looking in MRTK without needing to write custom code, and without using eye gaze as an input method.

To reproduce

  1. Disable Eye Tracking in the GazeProvider so that for input, the system is using head gaze instead of eye gaze.
  2. In your code, call InputRayUtils.TryGetEyeGaze or , grab eye gaze info from EyeGazeProvider.
  3. Deploy to Hololens 2, with eye tracking enabled.

Expected behavior

I expect to both InputRayUtils.TryGetEyeGaze or , EyeGazeProvider direction to return my eye gaze, since eye tracking is enabled.

However, the system actually returns the idrection of my head gaze, not my eye gaze.

I expect that if I am on a device that supports eye tracking, that MRTK has a built in way for me to access eye gaze data without needing to also use eye gaze as an input method.

Workaround

Currently to work around this issue it's possible to call the WinRT APIs directly, see EyeGaze.cs.

Screenshots

N/A

Your setup (please complete the following information)

N/A

Target platform (please complete the following information)

N/A

Additional context

Add any other context about the problem here.

sostel commented 4 years ago

The HoloLens 1 profile is probably missing the "Windows Mixed Reality Eye Gaze Provider": image

For additional info on how to access eye gaze while hand rays are active, you can also refer to this page: https://microsoft.github.io/MixedRealityToolkit-Unity/Documentation/EyeTracking/EyeTracking_TargetSelection.html#example-4-use-hand-rays-and-eye-gaze-input-together

Let me know if that helps!

julenka commented 4 years ago

Hi @sostel thanks for the answer. I think I misworded the bug -- the case I wanted was to use head gaze + airtap as the primary input method, but also know where the eyes are looking, which I do not think is possible. I will update the bug description.

sostel commented 4 years ago

Yes, that should work. In my prototype, I have a head gaze mode while still showing a custom eye gaze cursor. Let me double check that though.