microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6k stars 2.12k forks source link

MRTK3 Gaze input doesn't fallback to head gaze if eye tracking is disabled or not calibrated #11582

Closed AMollis closed 1 year ago

AMollis commented 1 year ago

This issue has been migrated a new MRTK repository, and the status of this issue will now be tracked at the following location:


Overview

MRTK3 Gaze input doesn't fallback to head gaze if eye tracking is not calibrated or eyes aren't being tracked

Repro

  1. Build EyeGazeExample scene with Eye Tracking permissions enable.
  2. Make sure you clear eye tracking calibration (you'll likely need to reset the device)
  3. Start app, and verify EyeGazeExample scene launches
  4. Move head around
  5. Box colors don't toggle to "red"

Expected

Box colors to toggle to "red" when head gazing (i.e. no eye gaze)

Details

The MRTK Gaze action mapping are currently using fallback composites. This mostly works, if the "eye controller" is not available, the system fallbacks back to head pose. However, there is a problem when "eye controller" is available but not tracking, the "eye controller" binding is still used, and the system doesn't fallback to head pose. The most common case of this happening is when HL2's eye controllers haven't been calibrated for the current user.

image

I've tried removing the fallback composite, but then only head pose is used, and eye controller is ignored.

image

I also tried using "isTracked" as a modifier. But this is not working either, no pose ends up being used. Image

AMollis commented 1 year ago

Fixed