Hand rays do not turn off when hololens keyboard / mixed reality keyboard is visible

[julenka]

Describe the bug

The mrtk hand rays do not turn off when the hololens 2 system keyboard is invoked. In contras. In the HoloLens shell, when the hand is near the keyboard, they turn off, when they are not near, they are on.

To reproduce

In the HandInteractionExamples scene, click on a text input field, or click the mixed reality keyboard field to get keyboard to show up. Observe that the hand rays do not turn off when the hand is near the keyboard.

Expected behavior

When hands are near the keyboard, the mrtk hand rays turn off.

[FreakTheMighty]

@julenka is this issue just visual or does it relate to controlling what gestures are available when the keyboard is open. I think I'm finding that I frequently accidentally dismiss the keyboard when an airtap gesture is detected. I have to put my thumbs away while typing or the keyboard will close unexpectedly.

[wiwei]

I don't think there's an API that lets an application know where the hololens keyboard is, so I don't think that we can really do anything here involving turning off hand rays when the hands are near the keyboard (i.e. it's impossible to get that information from within an app)

[wiwei]

The best we could do here is turn off hand rays when the keyboard is visible anywhere - @julenka what's your gut on which is better? (i.e. turn of hand rays when keyboard is visible, vs leave hand rays on)

[stale[bot]]

This issue has been marked as stale by an automated process because it has not had any recent activity. It will be automatically closed in 30 days if no further activity occurs. If this is still an issue please add a new comment with more recent details and repro steps.

[stale[bot]]

This issue has been marked as stale by an automated process because it has not had any recent activity. It will be automatically closed in 30 days if no further activity occurs. If this is still an issue please add a new comment with more recent details and repro steps.

[ArDevKarl]

I'm using this very ugly workaround in my prototype, to dsiable the pointers while the Keyboard is visible:

void Update()
{
    PointerUtils.SetHandRayPointerBehavior(!TouchScreenKeyboard.visible? PointerBehavior.Default : PointerBehavior.AlwaysOff);
}

[IssueSyncBot]

We appreciate your feedback and thank you for reporting this issue.

Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.

Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.

Please note that MRTK3 was released in August 2023. It features an all new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visithttps://www.mixedrealitytoolkit.org.

Thank you for your continued support of the Mixed Reality Toolkit!