search

microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6.02k stars 2.12k forks source link

Keyboard on the Hololens 2 not working #10025

Closed Daniel4144 closed 3 years ago

Daniel4144 commented 3 years ago

Describe the bug

The MixedRealityKeyboard is not showing up on the Hololens 2. I added a MixedRealityKeyboard to the scene and called ShowKeyboard("text"). The first call of ShowKeyboard throws an exception: ArgumentOutOfRangeException: Selection is out of range. Parameter name: selection at UnityEngine.TouchScreenKeyboard.set_selection (UnityEngine.RangeInt value) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.MixedReality.Toolkit.Experimental.UI.MixedRealityKeyboardBase.ShowKeyboard (System.String text, System.Boolean multiLine) [0x00000] in <00000000000000000000000000000000>:0

To reproduce

Steps to reproduce the behavior:

  1. Add a MixedRealityKeyboard component to the scene
  2. call ShowKeyboard
  3. See error, keyboard is not visible

Your setup

Target platform

Additional context

I also tried TouchScreenKeyboard Open("text"), which also did not work and throws no exception.

I have read, that the system keyboard on the Hololens is only working, if build type is set to XAML, but XAML builds are not working at all (no error, but nothing is visible on the Hololens). Because that thread is quite old and the build type is not mentioned in the documentation I am not sure, if that is still true.

Is a specific platform capability needed for the keyboard?

keveleigh commented 3 years ago

I have read, that the system keyboard on the Hololens is only working, if build type is set to XAML, but XAML builds are not working at all (no error, but nothing is visible on the Hololens).

This was true a long time ago but is no longer true. XAML builds not working is unexpected though.

Adding @MaxWang-MS for the other keyboard questions.

MaxWang-MS commented 3 years ago

@Daniel4144 The issues you encountered are not expected. I just tested both D3D and XAML builds with Unity 2020.3.12 and things worked without any problem. The fact that you called Unity's API directly (TouchScreenKeyboard.Open) but it did not work seems to me that you are either seeing a Unity bug or there could be some setup problem. That said, I cannot think of any special setting that is required for keyboard to work. Can you try again with Unity 2020.3 again and let us know if that solves the problem? Might also worth filing a Unity bug if the same Unity API call worked in 2020 but not 2021. By the way we are not officially supporting Unity 2021 at this time.

Daniel4144 commented 3 years ago

Thank you for testing, I also tested with Unity 2020.3.12 now (both D3D and XAML); same results. The ArgumentOutOfRangeException only appears, if ShowKeyboard() is called with a non-empty string.

Since I only need to type numbers, I could probably create a simple custom numpad as a workaround. I appreciate any other ideas what else I could try to make it work.

Regarding the XAML build: Although Unity reports the build process to be successful, there is an error in a post process build script of the MR OpenXR package (MixedRealityBuildProcessor.cs line 182) FileNotFoundException: Could not find file App.cpp

MaxWang-MS commented 3 years ago

Your results are definitely unexpected. Can you send us a minimum repro project so that we can take a look? I also encountered the error when building with XAML but I was able to ignore it, open the VS solution and build successfully. (@keveleigh do you know what's going on with MR OpenXR?) Also this is highly unlikely to be the issue, but can you make sure your HoloLens 2 has the latest updates installed?

keveleigh commented 3 years ago

Although Unity reports the build process to be successful, there is an error in a post process build script of the MR OpenXR package (MixedRealityBuildProcessor.cs line 182) FileNotFoundException: Could not find file App.cpp

Looks like a bug in the Mixed Reality OpenXR Plugin package. I'll get that filed and fixed.

MaxWang-MS commented 3 years ago

After changing the setup on my side I reproduced the exception being thrown issue. However this is a Unity bug and I have filed the bug to Unity (will post the issue tracker link here after it gets created). Strangely, the bug only appears when the scene/scripts are set up in particular ways so I was not able to repro with the modified hand interaction example scene. @Daniel4144 There are two potential ways to workaround the bug:

  1. Comment out keyboard.selection = new RangeInt(Text.Length, 0); in line 263 of MixedRealityKeyboardBase and see if that solves the problem.
  2. Use NonNativeKeyboard provided by MRTK. There is an sample scene demonstrating it.
Daniel4144 commented 3 years ago

@MaxWang-MS The Hololens is already on the latest version "Windows 21H1, Build 20348.1007", but I also tested with an older version before.

I created a minimum repro project to share and when I tested that project, the keyboard suddenly worked! Now also the project I was testing before works. I changed nothing on the old project, so I have no clue what solved the issue. Sorry for the waste of time and thank you for helping.

keveleigh commented 3 years ago

Although Unity reports the build process to be successful, there is an error in a post process build script of the MR OpenXR package (MixedRealityBuildProcessor.cs line 182) FileNotFoundException: Could not find file App.cpp

Looks like a bug in the Mixed Reality OpenXR Plugin package. I'll get that filed and fixed.

I've fixed this on the plugin side and it'll be in our next release!

MaxWang-MS commented 3 years ago

This issue is now tracked via #9056 and https://issuetracker.unity3d.com/issues/hololens-2-set-selection-throws-argumentoutofrangeexception