search

microsoft / MixedReality-HolographicRemoting-Samples

Code samples for using Microsoft's Holographic Remoting library.
Other
141 stars 48 forks source link

Hololens remoting player shows just "receiving..." #50

Closed IngGabriela closed 1 year ago

IngGabriela commented 3 years ago

Hello! I am new with unity and I have an Hololens 2 but I can not use Playmode. When I hit Play button, Holographic Remoting Player from Hololens shows "receiving" but nothing happens and unity shows just a black screen.

lappelsmeier commented 3 years ago

Hi @IngGabriela,

can you have a look at the documentation here: https://docs.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/features/tools/holographic-remoting?view=mrtkunity-2021-05 And see if checking your setup against the steps listed there helps?

Bye, Marc

IngGabriela commented 2 years ago

Nop, it doesn't...

vimusc commented 2 years ago

Hi @IngGabriela,

could you please check if: Unity -> Project Settings -> Xr Plug-in Management -> "Initialize XR on Startup" is enabled?

If this doesn't help, could you give us some information about your setup:

You can also look into the holodevelopers Slack for further help: https://holodevelopersslack.azurewebsites.net/

Best, Vincent

IngGabriela commented 2 years ago

Hi @vimusc My version is 2020.3.20f1. I use OpenXR, Mixed reality toolkit ad yes "Initialize XR on Startup" is enabled. Also thank you very much for the link :D Have a wonderful day, Gabriela

vimusc commented 2 years ago

Hi @IngGabriela,

unfortunately we are unable to reproduce this behavior. Using Unity (2020.3.20f1) with the OpenXR plugin (1.1.2) and the latest Holographic Remoting Player (2.7.1.0), works fine for us. Are you still having this issue?

Best wishes, Vincent

lekhachoa commented 2 years ago

Hello I try to use Holographic Remote many times but until now, it still not work on my PC and Laptop. I use Unity 2020.3.20f1, OpenXR Plugin (built in) v1.3.1, Mixed Reality OpenXR Plugin v1.2.1, and use Holographic Remoting Player app v2.7.2.0 from Windows Store . These are my configuration:

alt text

alt text

alt text

alt text

I tested it on my both PC (cpu i5 8250U), and laptop ( AMD R5 3500U) (Both no external Graphic Card). On my Laptop, it runs only on Unity Editor, when my head moves, the scene on Unity moves too, but on Holographic Remoting Player app on HL2 it shows always "Receiving.." On my PC, it does not even run.. I tested on other old PC with CPU 2500K and GPU 1030, It shows that, encode videos is not supported, although I did installed H265 code on PC (And tested by playing a h265 movie)

So, for the problem "Receiving..", what could I do to fix it?

I took the configuration form this repo. Just updating Unity and packages: https://github.com/LocalJoost/HandCollider/tree/openxrremotingsetup

vimusc commented 2 years ago

Hi @lekhachoa,

we will try to reproduce this once more with the additional information you provided.

Have you tried to select the H264 video codec: 68747470733a2f2f692e6962622e636f2f6772505a564b7a2f43617074757265342e706e67 If yes, does it work with H264?

Best, Vincent

lekhachoa commented 2 years ago

Hi @vimusc

Thank you very much indeed for your reply. I did test with H264 but it doesn't work. Turn out the problem is, all of my VGA are so weak, or do not support to encode h265,h264. I upgraded to new VGA and It works. Maybe the minimum requirement for VGA should be 3000 on benchmark scale of videocardbenchmark.net. Thank you for your reply again.

Until now I have not been able to test further yet. But could I use Hololens Remote App on PC to transfer holographic to Hololens and at the same time use Hololens to detect QR code direct on HL2? or how can I detect QR Code while using Holographic Remote?

Have a good day, Hoa Le

vimusc commented 2 years ago

Hi @lekhachoa,

great to hear that it works with the upgraded hardware. We will add a page about the system requirements to our documentation.

You can use the Microsoft.MixedReality.QR sdk to detect QR codes. This sdk also works with Holographic Remoting. You will find more information about QR code tracking here and we also have a QR code sample here.

Best, Vincent

vimusc commented 2 years ago

As promised: We added a page about the system requirements to our documentation. You can find it here.

IanMcAtee commented 2 years ago

Hi there, I'm experiencing a similar issue and could use some help troubleshooting.

ISSUE: Upon clicking the play button in the Unity Editor, the Holographic Remoting Player on the HoloLens 2 displays "Connecting..." for a few seconds, and then returns to the main "Waiting for Connection" screen, and the Unity Editor enters play mode on the PC. No remote content is ever rendered on the HoloLens.

EXPECTED BEHAVIOR: Upon clicking the play button in Unity, the scene should play remotely on the HoloLens.

HARDWARE, SOFTWARE, AND SETTINGS USED: Hardware:

Software:

Unity Settings:

UNITY WARNINGS THROWN: Warnings

SOULTIONS TRIED:

I can't seem to find what is going wrong. It may have something to do with the "Tracking origin unable to be set" and "Failed to calculate boundary bounds" warnings that Unity is throwing.

Any help that you guys can give me would be much appreciated. Also, please let me know if you require any additional information to help diagnose the problem.

lappelsmeier commented 2 years ago

Hi @IanMcAtee ,

thanks for the detailed report. One additional question from my side: Is this system a laptop (i.e. having both an active integrated and dedicated GPU)? We're currently looking (and fixing) some problems in this special case.

Bye, Marc

IanMcAtee commented 2 years ago

@maapp

Thanks for looking into this, but the system is not a laptop.

-Ian

IanMcAtee commented 2 years ago

SOLVED @maapp Thanks for looking into this, but I eventually found the solution. It was a frustratingly simple solution. My graphics drivers were one update out of date, and updating them solved the issue.

dfields-msft commented 2 years ago

FYI, I was seeing similar issues when trying to use Holographic App Remoting with my Surface Laptop, and they have been resolved after updating the Player app on my HoloLens2 to version 2.7.5 which was recently released. The update happened automatically, so this might "just work" if it was broken for you and you try it again now...

chenritiana commented 2 years ago

I also meet this program for a week ,when i used Windows Mixed Reality,i'll see the "receiving..."in hololens,when i used openXR and after play,unity will send an error like the follow . i have seen a video uploaded on youtube by Dilmer Valecillos,but also didn't work. image

vimusc commented 2 years ago

Hi @chenritiana,

thanks for reaching out to us, your issue looks different from the previous in this thread. Could you please create a new issue and provide similar information as in https://github.com/microsoft/MixedReality-HolographicRemoting-Samples/issues/50#issuecomment-1058383829 so that we can validate that your unity project was setup correctly.

Thanks in advance!

laultman commented 2 years ago

I have the same issue. I am using HoloLens 2 player with Holographic remoting Store app installed today. I am using Surface Pro 7, Intel i7 processor. I have Visual Studio 2022 installed and upgraded the solution to target current. I did not change the ISO C++17 as that breaks the code in OpenXrProgram.cpp line 185: std::string grammarFilePath{filename.generic_u8string()}; // non-unicode is supported up to ISO C++17 Standard (/std:c++17) . I am using the remote desktop native sample. I included the ip address of my HoloLens 2 device as the first arg to the app. The application compiles with no errors. At runtime I press the space bar as directed in the app. A few moments later the HoloLens 2 player app begins "Receiving...". The PC (Surface Pro) app screen is black. Not sure if it is supposed to be that way, but it does not render anything in the player. It also does not respond to keyboard "d" key to end program. In fact the loop never responds to any keyboard event after the space bar key event. My guess is that it is failing before it can properly initialize the application fully.

The code is out of the box, no modifications save this single change to fix its compile in VS 2022: XrMath.h line 108: //#ifdef _MSC_VER changed to

if (_MSC_VER <= 1929) // JLA compatible with VS 2022

The effect is to ignore the template macro.

When tracing some of the code I find that the CreateCubeGraphics() method returns "empty" in line 34 of App.cpp: auto program = sample::CreateOpenXrProgram(ProgramName, std::move(graphics), options);

Would be great to find the issue so I can begin to experiment with remote rendering. I have a use case with a government client that does not have access to Azure Remote Rendering because it is not available in Azure Government Cloud.

laultman commented 2 years ago

As stated above, I have this issue. I don't have VS 2019 and I don't have the ability to choose any install version I may wish as our government customer dictates what we use. That said, I need to know how to get the code to run with Windows 11 current version on a Surface Pro 7, i7 with Intel's Iris GPU all fully updated on Enterprise edition of Windows and Visual Studio. The code connects to the HoloLens 2 player app and it does call the RenderFrame method once. Somewhere in the RenderFrame method it never returns to the main loop. The app is effectively hung somewhere. It is not throwing an error in the debugger that is visible to me. I am seeing an m_ptr return in the generated base.h that is always null.

laultman commented 2 years ago

More information. Using the Windows Mixed Reality UWP solution works. I set the targeting to VS 2022 C++ and set the SDK to latest. No code changes at all. The issue I'm having is clearly related to the OpenXR implementation having issues.

laultman commented 2 years ago

Can't seem to find where Elvis leaves the building, but I know there is a loss of threads. I can close the app window while debugging and the app continues to run in the debugger. I can then force a BreakAll and the debugger shows the next line to execute when it returns. None of the application events are firing after the first call to render a frame. Whatever code that is maintaining a connection to HoloLens 2 player is still running even after the app window closes. There has to be a threading issue here. If I run the app from a command prompt window it connects as before with just the word "Receiving...". If I then close the application window it closes as expected. However, the app is still running. I can see it in the Task Manager. It is still holding the connection to the HoloLens 2 player even with the window closed. I have to then end the task in Task Manager to release the connection.

bethau commented 2 years ago

@laultman thank you for your issue report.

In our experiences streaming from a device that does not have a dedicated GPU leads usually leads to a bad user experience, as these devices don't have enough compute power. As such we would recommend that you use a device for development that has a dedicated GPU.

We still want to look into the issue you are facing. For this we need your help. We need you to collect information on what exactly happens on your machine before the black window appears. For this please follow these instructions: 1) Make sure you have the Windows Performance Toolkit installed (https://docs.microsoft.com/en-us/windows-hardware/test/wpt/) 2) Download this file: https://github.com/Azure/azure-remote-rendering/blob/master/Tools/ETLProfiles/AzureRemoteRenderingNetworkProfiling.wprp 3) On an admin cmd in the folder you placed the AzureRemoteRenderingNetworkProfiling.wprp into run wpr -start AzureRemoteRenderingNetworkProfiling.wprp -filemode 4) Now run the App 5) Wait until the "Recieving..." window appears and is black 6) On the admin cmd run wpr -stop trace.etl 7) Compress & upload the trace.etl file and send the resulting link to the e-mail address given in my github profile.

Be aware that the wpr tool will not only capture information about the single App process, but about everything happening on your system. Please do not run any workloads on your system the process sensitive data while running the above wpr commands.

bethau commented 2 years ago

@laultman Thank you for the trace. From the trace it looks like the application is stopping abnormally during initialization. Are you seeing a crash of the app?

One more thing you could also try is to disable the depth stream. For that modify the remote sample https://github.com/microsoft/MixedReality-HolographicRemoting-Samples/blob/main/remote/uwp/SampleRemoteApp.cpp#L704 And make sure ConfigureDepthVideoStream is called with DepthBufferStreamResolution::Disabled Also hard code m_canCommitDirect3D11DepthBuffer=false (https://github.com/microsoft/MixedReality-HolographicRemoting-Samples/blob/main/remote/uwp/SampleRemoteApp.cpp#L90)

vimusc commented 2 years ago

Hi @laultman, for the issue you described here https://github.com/microsoft/MixedReality-HolographicRemoting-Samples/issues/50#issuecomment-1173228402 you could try as a temporary workaround to turn of the frame mirroring extension by commenting out the following line: https://github.com/microsoft/MixedReality-HolographicRemoting-Samples/blob/b146cb9ee9ab68ed5a11666693126ca23bf4c337/remote_openxr/desktop/OpenXrProgram.cpp#L270 With this you will get intentionally a black/blue screen on the remote app but you should see content on the player. Also, the application should close as expected.

Thanks in advance and let us now if this works for you.

lappelsmeier commented 1 year ago

Closing for now since it's been a while since the last activity. Let us know if the problem persists.

laultman commented 1 year ago

Thanks for the assistance, from the comments and suggestions I was able to determine the conflict was in my thread handling where an unhandled exception occurred and never returned leaving the connection open but with no possible return.