PeerCCUnity Hololens app crashes after 2-4 seconds of video-call

[jairogloz]

I've finally managed to set the environment needed to test the PeerConnectionClient:

• PeerConnectionClient.WebRtc running on desktop
• PeerCCUnity app running on hololens
• peerconnection_server.exe running on a Windows 10 Azure VM

I'm able to connect both the desktop and hololens clients to the signaling server, but when I start a video-call, after 2-4 seconds of receiving only audio, the debugger stops at line 68 of App.cs:

public void Run() { m_AppCallbacks.Run(); }

With the following error:

An unhandled exception of type 'System.AccessViolationException' occurred in PeerCCUnity.exe Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

I'm about to start trying to fix this bug, any help would be appreciated.

Thanks in advance.

[jamescadd]

I have not seen this error - are you able to reproduce it readily? Also what's your HoloLens OS version?

[jairogloz]

Yeah, it happens every time I start a call, the application just crashes after 2-4 seconds of receiving audio. Even the holographic projection stops working for a moment and then the app just closes.

My OS build is 10.0.17134.111.

Thanks.

[jamescadd]

Ok,maybe one of the newer updates has caused an issue. I'll retest in a couple of days and see if I can repro.

[jamescadd]

Hi, I just tried a fresh build from the branch here and it worked on the latest production HoloLens OS. Are you still hitting the same issue? If you are able to try this branch it may help (though they are nearly identical except for the tooling version): https://github.com/ortclib/ortclib-sdk/tree/Mosa/20180320-peercc-unity

[jairogloz]

I have not been able to build WebRtc with this branch. I tried building the solution using Visual Studio 2017 but it failed due to a Visual Studio Version related problem, then I saw that in the steps it's specified to use VS2015 so I installed it, but now when I try to build it asks for the Windows 10 SDK (10.0.14393).

I already have three versions installed:

• 10.0.10586
• 10.0.16299
• 10.0.17134

But I guess I need that specific version in order to build the solution so I'll install it manually (since installing via VS2015 fails due to newer versions already installed).

I'll do that and try to continue with the steps to build the app.

[jamescadd]

Yes you unfortunately need 14393 installed as well as of now. With the next update we can consolidate down to 16299 (min version required for HoloLens) and 17134 (required for the C++/WinRT tooling) but for now it's a bit of a mess. Let me know if that resolves the issue. I'm not sure if those steps are updated but when I build the branch referenced above I use VS2017 latest update. Apologies for the confusion this definitely needs to be cleaned up for the next update.

[alnegod1]

I had a successful video call with the PC client using HoloLens, but when I disconnected the Called, PC client's camera turn off,ButHololens'external camera wouldn't shut down. Have you ever had this problem, what can I do to turn it off?

[jamescadd]

@alnegod1 It certainly sounds like we have a bug with closing the camera based on what you and others have reported. I am trying to reproduce this so we can fix the issue. If you're able to reproduce it in a Debug build running on HL (make sure to enable mixed-mode debugging for full logs) it would help to see the VS output.

[jairogloz]

Yeah, that happens sometimes, we haven't been able to reproduce it consistently though, so we have no clear idea on what causes it.

This issue was not related to the camera problem and has been solved already actually. So for the sake of order and to help future developers find the right advice for the camera not closing issue, I will close this one and suggest you @alnegod1 to open a new one.

Thanks.

[pthtcordnry]

@JairoGLoz can I ask how you resolved the AccessViolationException? I am stuck getting the same problem. I can connect both to the server, as soon as I hit call the PC will load and show the local camera but after about 2 sec (and before either camera show on the hololens) the hololens will crash.

[pthtcordnry]

or @jamescadd do you have any insights? I've been battling getting this setup for about 2 days now and can only successfully call PC to PC

[jairogloz]

@pthtcordnry sorry for the late response, I've actually solved this problem by cloning a specific branch and in a specific way @jamescadd explained to me here, by doing so the AccessViolationException no longer appeared. I can't tell you why exactly it gets solved though. Hope it helps.

[pthtcordnry]

I've tried the steps listed in that post and that also didn't work for me..

[jairogloz]

@pthtcordnry are you using Unity 2017.4.1 as they instruct? And also, are you building the solutions by yourself or are you referencing the NuGet version? if you're building the solutions be sure to chose the same configuration (Debug or Release) for both WebRtc.sln and WebRtcUnityD3D.sln. You're probably doing it that way already, it may be obvious for an experienced developer, but it wasn't for me at the beginning and I spent some time figuring that out, that's why I'm asking...

[pthtcordnry]

I'm building the solutions myself. I'll try that, I think I built them the same but I don't recall. I also got passed the AccessViolation error but I now get a System.IO.FileNotFoundException for the Media reference in the conductor script.

[pthtcordnry]

I get these errors one right after another DllNotFoundException: Unable to load DLL 'MediaEngineUWP': The specified module could not be found. (Exception from HRESULT: 0x8007007E) at ControlScript.Plugin.CreateLocalMediaPlayback() at ControlScript.OnEnable() at ControlScript.$Invoke30(Int64 instance, Int64 args) at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64 args, IntPtr method) (Filename: Line: 0)

Exception thrown: 'System.IO.FileNotFoundException' in PeerConnectionClientCore.dll An exception of type 'System.IO.FileNotFoundException' occurred in PeerConnectionClientCore.dll but was not handled in user code The specified module could not be found. (Exception from HRESULT: 0x8007007E)

FileNotFoundException: The specified module could not be found. (Exception from HRESULT: 0x8007007E) at System.StubHelpers.StubHelpers.GetWinRTFactoryObject(IntPtr pCPCMD) at Org.WebRtc.Media.CreateMedia() at PeerConnectionClient.Signalling.Conductor..ctor() at PeerConnectionClient.Signalling.Conductor.get_Instance() at ControlScript.Start() at ControlScript.$Invoke34(Int64 instance, Int64 args) at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64 args, IntPtr method)

[jairogloz]

Ok it's good that you're moving on. Inside your Unity PeerCC project Assets you should find a folder called Plugins and the mentioned dlls (MediaEngineUWP and PeerConnectionClientCore) should be there, if they're not it could mean your getting an error while building webrtcunity3d3.sln and thus de dlls are not being build, or maybe you're not opening the right unity project (also happened to me). The right one is at common/windows/samples/PeerCC/ClientUnity/Unity/ and is called PeerCCUnity.

Also, once you build the project from Unity Editor, be sure to include the magic piece of code in the Package.appxmanifest.

[jamescadd]

Appreciate your help Jairo! I wanted to add a note that we understand that this is not straightforward for a Unity-first developer and are working to improve things. The most recent effort is a move to the 2018 Unity tooling which will include a project that is buildable/runnable from Unity, as well as a plugin for WebRTC UWP that we plan to make available in the Unity store. @pthtcordnry if you're still stuck please email my microsoft dot com address, jacadd, and we can setup some time to troubleshoot.

[jairogloz]

Appreciate your help Jairo! I wanted to add a note that we understand that this is not straightforward for a Unity-first developer and are working to improve things. The most recent effort is a move to the 2018 Unity tooling which will include a project that is buildable/runnable from Unity, as well as a plugin for WebRTC UWP that we plan to make available in the Unity store. @pthtcordnry if you're still stuck please email my microsoft dot com address, jacadd, and we can setup some time to troubleshoot.

A pleasure, just trying to give a little bit back 😃 good to know you guys keep working on making things easier for Unity developers.

[markgrossnickle]

@jamescadd Any update on

The most recent effort is a move to the 2018 Unity tooling which will include a project that is buildable/runnable from Unity, as well as a plugin for WebRTC UWP that we plan to make available in the Unity store

[jamescadd]

Hi @markgrossnickle - The latest on Unity is that the releases/m71 branch of https://github.com/webrtc-uwp/webrtc-uwp-sdk now includes Unity support with 2018.3 tooling. I was able to run it on a device yesterday with 2-way video. There's a known issue where H.264 video freezes almost immediately, but it has been resolved and we expect to have a PR to fix this next week. Regarding the plugin, keep an eye on MRTK v2 for developments over the next week or two. The work to land WebRTC there is looking pretty likely to succeed so if you prefer that approach you can get started now. https://github.com/Microsoft/MixedRealityToolkit-Unity/wiki/Getting-Started-with-MRTK-v2

[markgrossnickle]

Thanks @jamescadd! Definitely looking for a Unity first project rather than UWP that takes a unity component so MRTK would be ideal. I'm part of the MRTK moderators and knew they wanted to use WebRTC for sharing on their roadmap but didn't realize it was making headway. I'll check with them. Thanks!

[TiagoShen]

Hi @jamescadd ! I am trying releases/m71branch and wonder if the Unity project could run on PC too. Any suggestions? Is the building process similar to this? Thanks in advance.

[jamescadd]

Hi @TiagoShen - there have been a couple of extra fixes since last week. This branch is your best option and it does have updated documentation on how to build: https://github.com/webrtc-uwp/webrtc-uwp-sdk/tree/Mosa/m71-unity-fixes Currently there's a known issue with placing multiple calls due to the UI state, H.264 fix PR is not in yet (but hoping to get it today).

[jamescadd]

Please follow this branch for future Unity support in MRTK. This will be our recommended approach going forward for WebRTC in Unity. If this does not work for anyone please let me know: https://github.com/Microsoft/MixedRealityToolkit-Unity/issues/3590