SamplePlayer and SampleRemote (desktop) - unable to connect (fails with 'TransportConnectionFailed')

[philipjpratt]

Hello,

I checked the development pre-requisites, etc., and built the SamplePlayer and SampleRemote (desktop) solutions out-of-the-box. After connecting laptop and HoloLens2 over a local WiFi network, I spin up the HL2 app and launch the remote sample with the IP of the HoloLens as first command line argument. On hitting space bar, the client/server fail to connect. Running the remote end up in the debugger, I see the disconnect event is fired with code 'TransportConnectionFailed'.

First thing I tried was to disable the firewall (server side), but no change in behaviour. Maybe I've missed something obvious? Some tips for troubleshooting would be very useful... Is there a minimum HL2 OS version required, for example?

As an aside, I presume it would be possible to port the C++ Player, or at least the fundamental parts of it, to C#...? My existing HL2 app is written in C# (using SharpDX, etc.), and I'm hoping to integrate remoting, rather than do the port the other way around.

Thanks in advance for your help.

Kind regards, Philip

Dr Philip J Pratt

[vimusc]

Hi @philipjpratt,

thanks for reaching out to us. As you asked for troubleshooting and system requirements here are links to our documentation just in case you haven't seen it already: Troubleshooting, System Requirements. We don't have an explicit minimum version for the HL2 OS but in general, keeping all involved parts (OS, drivers, ...) up to date is recommended.

Could you please give us some information about your setup:

• Which Holographic Remoting version is used?
• Which hardware is used on the remote side?
• Do you use "Windows Mixed Reality" or "OpenXR"?

I would recommend to use our store player first to reduce the possible sources of error. Does it work with the store player?

Do you have a specific reason to port the player to C#?

Best regards, Vincent

[philipjpratt]

Hi Vincent,

Thanks for getting back to me promptly. I saw the troubleshooting guide and system requirements already. I upgraded one of my HoloLens2s to the latest OS build (20348.1503), but that didn't fix the problem. Also tried the 'store player', but observed the same behaviour.

In answer to your questions:

• I was using the head of the main branch as of this morning (should I have used the head of the Version_2.8.0 branch instead, in case some further development has broken something?)
• I was using the "Desktop" version of the "Remote" app, i.e. not the OpenXR port
• On the remote side, I'm using a Lenovo W541 laptop (Win10 Pro 21H2 OS build 19044.1706), with a wired connection to a NetGear AC2300 (model R7000P) 5G WiFi router. Tried with Windows Firewall switched off also.

I have quite a substantial code-based in C# for my main HoloLens2 app, and porting the other way would take a very long time. I tried adding the 2.8.0 remoting NuGet package into a vanilla C# HL2 sample app, but can't locate the symbols/APIs anywhere...

Do let me know if I can provide any further diagnostic information.

Kind regards, Philip

[vimusc]

Hi Philip,

thanks for the reply.

The main branch is the correct one. I tested it on my side with the same OS versions, 2.8.0.0 store player and 2.8.0.0 desktop remote sample which worked fine.

Do the two devices (HoloLens2 and your laptop) see each other in the network? Can you for example access the Windows Device Portal over Wi-Fi? For firewall settings keep in mind that if you use the store player, the player is the server and the remote application is the client (for more info see HAR-Terminology)

As an aside, for most developers our store player is sufficient and there is no need for a custom player except you would like to use some enhanced remoting features. All the app logic runs on the remote side. Which means that most likely all of your existing logic will end up in the remote application not on the player side.

Best regards, Vincent

[philipjpratt]

Many thanks, Vincent.

Yes - the devices can see each other. In fact, I installed the sample player via the Windows Device Portal (over WiFi), having published an AppPackage for it - rather than doing so via Visual Studio. I'll give it another go and, if necessary, will fire up a packet sniffer (e..g 'pktmon') in the hope that this will shed some light on the problem.

You're almost certain right that in going down the remoting route, the app logic will end up on the remote side. But I guess I'm looking for a half-way-house in the interim - it would take me a very long time to port the existing app. So I'm still interested in the question - can one use the new/current NuGet remoting package in conjunction with C# holographic development? Perhaps this is only possible if one were to write wrappers, or maybe it's simply impossible at this stage?

The other question related to what I'm trying to achieve is this - can I integrate CUDA with the remote app, such that I can ray trace directly into the back buffers? I'm taking steps to test this myself, but first need to upgrade my hardware!

Kind regards, Philip

[vimusc]

Hi Philip,

our NuGet packet is for native C++ so yes, you have to write some wrappers to access it with your C# app. You could write a C++ wrapper around our NuGet package. The wrapper can contain only the subset of the functionality you need. You could then export this wrapper as your own DLL and P/Invoke in this DLL from your C# app.

If you decide to use the newer OpenXR API it's maybe also worth looking at Silk.NET which provides bindings to OpenXR. It doesn't provide bindings for HAR but maybe its a good starting point.

CUDA should work, you can access the DirectX resources on the remote side and as far as I know CUDA has an interop to DirectX.

Best regards, Vincent

[philipjpratt]

Thanks, Vincent.

I've given your suggestion some thought, however there are a couple of situations where I can't quite see how one would implement the wrappers. For example, the UpdateUserSpatialFrameOfReference method of winrt::Microsoft::Holographic::AppRemoting::PlayerContext takes an instance of winnt::Windows::Perception::Spatial::SpatialCoordinateSystem. I'm not sure how one would marshal such an instance in the context of the P/Invoke solution. Maybe you could elucidate a little further?

In parallel, I've considered creating a C++/CX Windows Runtime Component, which pulls in the Remoting nuget package itself. However, it's not clear how one would marshal instances across the 'Platform' boundary in this setting either - at least not to me! It may be the case that I just have to build a new remote player as a native C++ app, but I want to avoid that if possible (and not too involved), so as to extend, rather than replace, my existing C# app.

I've still not been able to get the player and remote halves to talk to each other. The only component I've not switched out is the laptop on which I'm running the (desktop) remote app. I will be acquiring a new box soon with a fresh install of Windows 10, etc., and will repeat the test - it's possible there's something amiss with my aging laptop, e.g OS/software, as a hardware problem is unlikely.

With thanks again for your continuing help, Philip

[vimusc]

Hi Philip,

sorry for the late response I was on vacation.

A wrapper would need to fully decuple remoting and the Windows Mixed Reality API in order to work. I guess this would only work well for an existing application if there is already some abstraction around the WMR API and even then, there are maybe some hidden pitfalls. I also had a look at the C++/CX Windows Runtime Component but unfortunately can't assist you there. The cleanest way would be native C++ but I can fully understand that you don't want to replace your existing app.

Are you using H265 (with the installed extension on the remote machine) or H264?

Best regards, Vincent

[philipjpratt]

Hi Vincent,

No problem - I trust you had a good break. Thanks for getting back to me.

For now and just for the purpose of prototyping volumetric rendering, I've decided to go with a native C++ project on the remote side. I figured I should take the shortest path to seeing how it's going to look and perform, and then if worth taking further, make the necessary choices around porting/replacing existing code.

Re compression, etc., I'm using whatever comes out of the box, and am running on a brand new box with a fresh Windows 11 install. Do I need to install a component explicitly (for remoting)? Is there a way to check which one is being used, currently, in this context? And do you find that there's a noticeable performance difference between the two?

Thanks and speak soon, Philip

On Wed, 29 Jun 2022 at 17:51, Vincent Musch @.***> wrote:

Hi Philip,

sorry for the late response I was on vacation.

A wrapper would need to fully decuple remoting and the Windows Mixed Reality API in order to work. I guess this would only work well for an existing application if there is already some abstraction around the WMR API and even then, there are maybe some hidden pitfalls. I also had a look at the C++/CX Windows Runtime Component but unfortunately can't assist you there. The cleanest way would be native C++ but I can fully understand that you don't want to replace your existing app.

Are you using H265 (with the installed extension on the remote machine) or H264?

Best regards, Vincent

— Reply to this email directly, view it on GitHub https://github.com/microsoft/MixedReality-HolographicRemoting-Samples/issues/59#issuecomment-1170233598, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQ5Y3XBEFOGGHKCMX5C7E7TVRR5I3ANCNFSM5W4VFBJQ . You are receiving this because you were mentioned.Message ID: <microsoft/MixedReality-HolographicRemoting-Samples/issues/59/1170233598@ github.com>

[vimusc]

Hi Philip,

great that you will try it with native C++.

With the Windows Mixed Reality (WMR) API you can configure the used video codec within the CreateRemoteContext() call. With WMR we use H265 only if it is available. If you (at some point) use the newer OpenXr API H265 is our default video codec in the samples. We have a link to the video codec extension here this extension is needed in order to make use of H265. As you are going to look how it performs, H265 is recommended as it will give the same video quality with lower bitrates. You can also increase the maximum bitrate within the CreateRemoteContext() call to increase the visual quality.

Regarding your new computer, here are our System Requirements, if you start with our samples you don't need to add an additional component explicitly for remoting. Otherwise, our NuGet packet is required.

Best regards, Vincent

[philipjpratt]

Thanks, Vincent - that's helpful.

The new computer is quite high-end and has an RTXA6000 GPU :-) I'm very keen to see what I can make it do (in terms of volumetric rendering) when combined with the HoloLens/Remoting.

All best wishes, Philip

On Thu, 30 Jun 2022 at 12:29, Vincent Musch @.***> wrote:

Hi Philip,

great that you will try it with native C++.

With the Windows Mixed Reality (WMR) API you can configure the used video codec within the CreateRemoteContext() call. With WMR we use H265 only if it is available. If you (at some point) use the newer OpenXr API H265 is our default video codec in the samples. We have a link to the video codec extension here https://docs.microsoft.com/en-us/windows/mixed-reality/develop/native/holographic-remoting-troubleshooting#h265-video-codec-not-available this extension is needed in order to make use of H265. As you are going to look how it performs, H265 is recommended as it will give the same video quality with lower bitrates. You can also increase the maximum bitrate within the CreateRemoteContext() call to increase the visual quality.

Regarding your new computer, here are our System Requirements https://docs.microsoft.com/en-us/windows/mixed-reality/develop/native/holographic-remoting-system-requirements, if you start with our samples you don't need to add an additional component explicitly for remoting. Otherwise, our NuGet packet is required.

Best regards, Vincent

— Reply to this email directly, view it on GitHub https://github.com/microsoft/MixedReality-HolographicRemoting-Samples/issues/59#issuecomment-1171101817, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQ5Y3XF72IZTNHS5KLKCAVTVRWAJDANCNFSM5W4VFBJQ . You are receiving this because you were mentioned.Message ID: <microsoft/MixedReality-HolographicRemoting-Samples/issues/59/1171101817@ github.com>

[laultman]

@philipjpratt Following along here, having some issues similar to yours. You and I are traveling along the same path it seems with existing code base and needs for interaction with the HL2 device at a deeper level. I am having issues it seems with versions of VS libs between this code and newer releases of VS. It is a pain to try to get all these different parts together when everybody is on the bleeding edge. Anyway, could we connect elsewhere? My LinkedIn name is laultman.

[chairobl]

Closing this ticket as the original problem seems to have been resolved and there hasn't been activity in some time now. @philipjpratt , @laultman feel free to reopen this if you're still encountering problems with this :)