search

microsoft / MixedReality-WebRTC

MixedReality-WebRTC is a collection of components to help mixed reality app developers integrate audio and video real-time communication into their application and improve their collaborative experience
https://microsoft.github.io/MixedReality-WebRTC/
MIT License
909 stars 283 forks source link

Unity App Crashing in HoloLens 2 when connecting to PC #684

Open SheehanC1 opened 3 years ago

SheehanC1 commented 3 years ago

Description: I followed the Unity Hello World tutorial for using MR WebRTC with Unity. I got it to successfully connect 2 PCs, but when I moved on to try a connection that's HoloLens and PC, the build in the HoloLens crashes the moment both are connected to the node-dss server running on the PC.

Steps To Reproduce

  1. Set up the node-dss serve to run on the PC using Node.js set DEBUG=dss* npm install npm start
  2. Press the play button in Unity (I'm using Unity 2019.3.13) to run the setup for the PC making sure the Remote and Local Peer Ids are correct for it.
  3. See that it is displaying the local webcam capture from the PC on one of the two screens.
  4. Check the Node.js command prompt window to see that the game running in Unity has connected.
  5. Run your build in the HoloLens 2 (having already checked its Local and Remote Peer Id setup is all correct before building and placing on the headset)
  6. See issue: The game in the HoloLens opens briefly only to crash a split second later.

Expected behavior In the headset, you should see 2 screens, one displaying the local webcam capture of what the HoloLens is seeing, and the other screen displaying what the PC cam is seeing remotely. On the PC you should see something similar but with the local and remote swapped.

Screenshots Here is a screenshot of the errors/SDP messages logged the moment it tries to connect and immediately crashes Screenshot (26) Screenshot (27)

Environment

Additional context The game in the Hololens only crashes if the PC is also running its version of the game/in Unity and the Node.js is running the node-dss server to connect them. If the node-dss-server isn;t running or the game isn't running on the PC, it just captures and shows the Local video only (until the conditions are met for it to crash).

From what I can tell, it might be associated with the Preferred Video Codec where the PC is using VP8 or VP9 cause it has to and the HoloLens is using H264? similar issue maybe

ANG0-0GNA commented 3 years ago

I have the same question. The unity project can worl well on two PCs, but it doesn't work on a hololens 2 and a PC. Has there been any progress on this issue? Thank you!

ANG0-0GNA commented 3 years ago

I have the same question. The unity project can worl well on two PCs, but it doesn't work on a hololens 2 and a PC. Has there been any progress on this issue? Thank you!

It works after importing mrtk.

SheehanC1 commented 3 years ago

Hi @ANG0-0GNA. What versions of the MR-Web RTC, MRTK, and Unity did you use? And is your scene with the WebRTC stuff in it configured for MRTK as well? If so, did you have to change any of the MRTK configs for it to work? Thanks for any insights you can give!

ANG0-0GNA commented 3 years ago

hi @SheehanC1. webrtc: 2.02, mrtk: 2.6.1. I use the VideoChatDemo in webrtc.samples, and import the mrtk to VideoChatDemo.

SheehanC1 commented 3 years ago

Thanks @ANG0-0GNA!

xuhaozheng commented 3 years ago

Hi, @ANG0-0GNA Did you directly import MRTK to a project which contains VideoChatDemo? Did you import both WebRTC and MRTK via MixedRealityFeatureTool? Have you ever encounter any Unity Crash?

ANG0-0GNA commented 3 years ago

Hi, @ANG0-0GNA Did you directly import MRTK to a project which contains VideoChatDemo? Did you import both WebRTC and MRTK via MixedRealityFeatureTool? Have you ever encounter any Unity Crash?

@xuhaozheng Yes. Unity may crash.

xuhaozheng commented 3 years ago

@ANG0-0GNA So how did you make it runnable? Build the unity project and deploy it to Hololens directly?

I have the same question. The unity project can worl well on two PCs, but it doesn't work on a hololens 2 and a PC. Has there been any progress on this issue? Thank you!

It works after importing mrtk. How did you test and find it worked? I just tried to import MRTK to the VideoChatDemo and deploy it to the Hololens. It seems do not work.