How to use HoloLens's GPU

[nocardio]

Hi @cgsaxner, have a good day.

I have tried to develop CV-based application in my HoloLens 2, however the running time is far from my expectation. When I tried to debug via Device Portal, it seems that HL2 is automatically using its CPU to run the program and there is no GPU process at all.

In your paper Inside-Out Instrument Tracking for Surgical Navigation in Augmented Reality, it was mentioned that you were using HoloLens GPU to process the algorithm. Instead of using CPU, I am wondering if I also can use HL2 GPU to process my algorithm.

Could you please guide me on how to use HoloLens2 GPU? I tried to look at the official HL2 website, but I cannot find anything.

Thank you in advance :)

[cgsaxner]

Hello, you can do that using DirectX compute shaders. Essentially, you need to write an hlsl shader that carries out your computation, and on the C++ side, you need to compile the shader, grab the pixel data from the HoloLens frames, pack them into a directX texture, and provide these textures as input to the shader.

This is a good starting point: https://learn.microsoft.com/en-us/windows/win32/direct3d11/direct3d-11-advanced-stages-compute-create