Hololens2 performance issues

[iamx1212]

Hello, I used UnityVolumeRendering to load a CT (Dicom) image and deployed it to the Hololens2 real machine. After opening it, it was very stuck. The model can be displayed but when I look at the model, it becomes very slow and stuck. What should I do?

[iamx1212]

[mlavik1]

Hi @iamx1212 !

I don't have a Hololens device myself, but I know there have been other users using this plugin with Hololens, so I believe it should be possible to get acceptable performance.

First of all: Is this a publically available dataset that you can share with me?

If not, could you check how large the dimension of the dataset is? If it's very large, you could consider: A: Open the volume rendering settings window, enable "Show downscale prompt", re-import your DICOM and choose "yes" when asked if you want to downscale the dataset or B (better): Use some software (I think 3D Slicer can do that) to downsample the dataset.

That should reduce the graphics memory usage (and memory bandwidth usage), and probably improve performance.

Also, you might want to check the appearance settings on the VolumeRenderedObject:

• Enable ray termination: ON
• Disable cubic interpolation: OFF
• Disable Lighting: OFF (unless you can afford it - it's expensive though)

[iamx1212]

I have been trying to solve the lag problem recently, but it hasn't had a particularly good effect. [Uploading CT.zip…]()

[mlavik1]

Just a thought: Is the performance problem bad in all render modes? (Direct volume rendering vs isosurface rendering)? Maybe not so much in Maximum Intensity Projection mode?

I saw that Hololens 2 uses a resolution of 2880x1406, which might make volume rendering a bit tricky - considering its GPU. You could try change the resolution, or in Unity: UnityEngine.XR.XRSettings.renderScale = 0.5f or something. Might look really bad, but would be interesting to see how much it affects the performance.

You could also try to change the MAX_NUM_STEPS values (one for each render mode) in the DirectVolumeRenderingShader.shader

[iamx1212]

Yes, there will be lag in volume rendering, maximum density, and isosurface drawing at MAX-NUM-STEPS 512. Lowering the MAX-NUM-STEPS value can slightly alleviate this. I used UnityEngine XR The XRSettings.renderScale=0.5f method can also alleviate lag, but my method may have been incorrect. When using it with Holo on my head, the entire model shook noticeably as I walked around. script.zip

[mlavik1]

@iamx1212 hmm, if even maximum intensity projection mode is slow, then I'm afraid it's not looking very promising.. Since I don't own a Hololens device it's hard for me to investigate if this is just due to the hardware, or if there are any performance issues, but I think I'm running out of suggestions now... Do you need to run the app natively on the Hololens, or would it be an option to run it through a PC's GPU? I don't have experience with Hololens, but on Oculus you have Link/Airlink for that. I saw there's something called "Holographic Remoting Player" for Hololens.

[DiangF]

Do you have any measures to address the issue of loading rendering on HoloLens? thanks

[mlavik1]

@DiangF Hmm, what do you mean with "loading rendering"? Are you running into any issues loading datasets?

@iamx1212 I've recently added a new "sampling rate multiplier" setting. If you check out latest from "master" branch you can try it out. Try reducing the value and see how much the FPS improve. Feel free to share here! :)

I now also do opacity correction, so the only visual different should now be more/less artifacts (the colours don't change like they did before)