How to access the PV camera in the player on the HoloLens with Holographic remoting

[yixun]

Hi, I need to access the PV camera of HoloLens to do some vision job with Holographic remoting. As I know I cannot access the PV camera from the PC side. I would like to know can I access the camera on the player side on the HoloLens?

Thanks. YL

[FlorianBagarMicrosoft]

Hi YL, camera access is possible as described here: https://learn.microsoft.com/windows/mixed-reality/develop/native/mixed-reality-capture-directx#built-in-mrc-photo-and-video-camera-access

With Holographic Remoting you need to build your own custom player which implements the camera access on top of the normal remoting stuff a player is doing. If you need the data that is detected by your vision job, you would need to implement a custom data channel to transmit that data between your custom player and your remote application.

[yixun]

Your provided link is very helpful. Thank you.

[yixun]

@FlorianBagarMicrosoft , as I know Holographic Remoting works by replacing the Windows Mixed Reality runtime which is part of Windows with a remoting specific runtime. I would like to know as I develop a customized player, will the remoting specific runtime impose some constrains?

[FlorianBagarMicrosoft]

Your statement is only true for remote apps, in this case Holographic Remoting completely replaces the Windows Mixed Reality runtime. In contrast to that a custom player does not replace the Windows Mixed Reality runtime shipped with Windows but instead injects an intermediate layer between the player app code and the Windows Mixed Reality runtime code.

This enables Holographic Remoting to transparently operate on the player side, say the only Holographic Remoting specific API a developer has to call while rendering frames is BlitRemoteFrame. Everything else in a player app while rendering is just normal Windows Mixed Reality API but Holographic Remoting is aware of call to like CreateNextFrame, TryGetViewTransform, etc. because of that intermediate layer.

To answer your question, no it should not.

[yixun]

From the link: https://learn.microsoft.com/en-us/windows/mixed-reality/develop/native/holographic-remoting-create-player I copy the warning section of the above link: "Holographic Remoting works by replacing the Windows Mixed Reality runtime which is part of Windows with a remoting specific runtime. This is done during the creation of the player context. For that reason any call on any Windows Mixed Reality API before creating the player context can result in unexpected behavior. "

I am confused because as you said the player does not change the runtime.

[FlorianBagarMicrosoft]

The docs are incorrect in this context, we will fix that in the next update. Thanks for pointing this out.