MRTK.update() takes 10-15 ms using HoloLens 2 default profile while a hand is in view.

[darax]

Describe the bug

I have been asked to investigate frame rate drops in an MRTK application. While doing so I made a brand new scene. I used the menu item to configure the scene for MRTK. I configured MRTK for HoloLens 2 with just a cube to the scene. I ran on my device and pointed the deep profiler at it. The amount of time being used by MRTK.Update() leaves very little time for an application to do its own per frame processing.

To reproduce

Steps to reproduce the behavior:

1. Make a new scene
2. Use the menu item Mixed reality Toolkit ->Add to scene and configure
3. In the hierarchy select 'MixedRealityToolkit'
4. In the inspector choose the 'DefaultHoloLens2ConfigurationProfile'
5. Build the app with deep profiling support
6. Run the app on a HoloLens2 device
7. Watch the profiler with and without a hand in view

Expected behavior

I'm not sure if MRTK has defined per frame budgets that I can affirm are being exceeded. That being said, there is so little room left with the default HoloLens 2 profile enabled it is hard to imagine being able to create anything beyond a basic application and achieve 60 FPS on HoloLens 2.

Screenshots

Your setup (please complete the following information)

• Unity Version 2019.4.6f1
• MRTK Version 2.5.1

Target platform (please complete the following information)

• HoloLens 2

[david-c-kline]

recreated the scenario using the latest mrtk_development (as of mid-day 2/9/2021).

• DefaultHoloLens2Profile
• Single cube in the scene, using MRTK standard shader based material
• Release build and deploy to device.

Frame rate is consistently 60fps +/- 2, per the diagnostic system's profiler. this is with 0 - 2 hands in view.

In discussing the issue with @darax, he indicated that there had been an apparent issue in MRTK where the quality level was not appropriately set for transparent displays that may have been fixed in a 2.5.x release.

Also, the overhead of profiling (esp. deep profiling) may be impacting the update time.

I will continue to investigate using the Unity profiler's standard mode (not deep profiling) and examine the logs.

[darax]

Thanks for the follow up, David. I would be interested to know how many milliseconds are being spent in the MRTK related functions. This helps us understand how much overhead is left to add our application's work on top of the work being done by MRTK.

[david-c-kline]

While profiling, I ran with one hand in view. In general, MRTK.Update was in the ~3ms range. When there were frames that took longer...

4.2 ms - WindowsMixedRealityArticulatedHand.UpdateController took 3.02ms total (for both hands)

8.24ms - WindowsMixedRealityEyeGazeDataProvider.Update took 4.97ms

[david-c-kline]

In the eye gaze provider example, there was a garbage collection that occurred. Some investigation should be performed (I will open a new issue to track) to investigate where the garbage originates.

[david-c-kline]

I would be interested to know how many milliseconds are being spent in the MRTK related functions.

The MRTK inner loop was instrumented for the Unity profiler in 2.4.0. If there is a key metric that is not appearing when you trace, please let us know and we will be sure to add the profiler markers.

[darax]

awesome! What did that instrumentation show during your analysis?

[david-c-kline]

Sorry i missed seeing your question @darax. There were some issues that were fixed in 2.4 and there was some data that pointed to issues with how the application being instrumented constructed their UnityUI panels that generated >1-2ms hierarchy walks

[stale[bot]]

This issue has been marked as stale by an automated process because it has not had any recent activity. It will be automatically closed in 30 days if no further activity occurs. If this is still an issue please add a new comment with more recent details and repro steps.

[IssueSyncBot]

We appreciate your feedback and thank you for reporting this issue.

Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.

Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.

Please note that MRTK3 was released in August 2023. It features an all new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visithttps://www.mixedrealitytoolkit.org.

Thank you for your continued support of the Mixed Reality Toolkit!