[Mosa/20180320-peercc-unity]: PeerCCUnity on desktop throws exception on TransferVideoFrame()

[DanAndersen]

I have a HoloLens Unity-based WebRTC application that is using the codebase in the Mosa/20180320-peercc-unity branch. This application is a copy of the PeerCCUnity project (same Conductor, Signalling scripts, etc) except that the UI elements appear in a full 3D HoloLens app rather than being in a 2D Hologram window. I can connect my HoloLens app to the normal PeerCC UWP XAML app (in the WebRtc.sln solution) and successfully receive MRC video from the HoloLens in the desktop PeerCC app (the video going the other way around has performance issues but that's for another day).

The issue I'm running into now is when I try to run PeerConnectionClientUnity (in the WebRtcUnityD3D.sln solution) as a Unity UWP app on my desktop, and connect to it from the HoloLens app. Both my HoloLens app and the desktop PeerConnectionClientUnity app can both connect to my signalling server, but when I try to press Call, I get a Platform::FailureException caused by the following line in MediaEnginePlayer.cpp:

MEDIA::ThrowIfFailed(
    m_spMediaEngine->TransferVideoFrame(m_primaryMediaTexture.Get(), &m_nRect, &m_rcTarget, &m_bkgColor)
);

The debug logs are attached at 2018-05-03_exception_peerccunity_desktop.txt

Has this issue been encountered before? I am using the same MediaEngineUWP plugin on my HoloLens app, which is not encountering this issue. Thanks in advance for any guidance you can provide!

[jamescadd]

@DanAndersen The error code in the logs 0x80004005 is a generic error so it's hard to diagnose with a repro. Have you tried with the latest code to see if this still occurs?

[fcagnetta]

@DanAndersen how did you get the stream from Unity app on Hololens to UWP app on desktop? I am having issues accomplishing this task. The only thing that works is: Unity app built for UWP-PC which stream video to UWP desktop app.

Thanks, Francesco

[DanAndersen]

@fcagnetta What are the specific issues you're having? I was having issues getting the latest version of the peercc-unity branch to load the Org.WebRtc library on the HoloLens, but found a workaround. See https://github.com/webrtc-uwp/webrtc-uwp-sdk/issues/50 .

[DanAndersen]

@jamescadd Back to the original error in this issue; I am still encountering the exception when running the Unity codebase on desktop (and when setting up a peer connection with a HoloLens instance):

Exception thrown at 0x74F6D722 (KernelBase.dll) in HoloPoseMentorSample.exe: 0x40080201: WinRT originate error (parameters: 0x80004005, 0x00000013, 0x2D6FF164).
Exception thrown at 0x74F6D722 in HoloPoseMentorSample.exe: Microsoft C++ exception: Platform::FailureException ^ at memory location 0x2D6FF644. HRESULT:0x80004005 Unspecified error
WinRT information: Unspecified error
Stack trace:
 >[External Code]
 >[Inline Frame] MediaEngineUWP.dll!MEDIA::ThrowIfFailed(HRESULT hr) Line 37
 >MediaEngineUWP.dll!MEPlayer::OnTimer() Line 900
 >[Inline Frame] MediaEngineUWP.dll!MEPlayer::RealVSyncTimer() Line 871
 >[Inline Frame] MediaEngineUWP.dll!MEPlayer::StartTimer::__l2::<lambda_e787bec0b7c5c524fddc749ba7e2674d>::operator()(Windows::Foundation::IAsyncAction ^) Line 828
 >MediaEngineUWP.dll!Platform::Details::__abi_FunctorCapture<<lambda_e787bec0b7c5c524fddc749ba7e2674d>,void,Windows::Foundation::IAsyncAction ^>::Invoke(Windows::Foundation::IAsyncAction ^ <__args_0>) Line 889
 >[External Code]

I'll continue to investigate on my end to see if I can figure it out too.

[DanAndersen]

Update: I'm still doing some debugging to figure out why the incoming video is not appearing in my desktop UWP Unity app, but I was able to get past the exception on TransferVideoFrame().

Apparently there have been reported issues with the TransferVideoFrame() function when running on some NVIDIA graphics cards (see here, here, and here). My dev machine has an NVIDIA Quadro K4000.

After installing the latest NVIDIA drivers (397.93 Quadro drivers in my case), this particular error was bypassed, and I am able to see that OnRawVideoFrame is being called from the remote peer video. The video isn't appearing in the Unity texture set aside for the incoming video, but I'll investigate further on that and maybe open another issue if needed.

EDIT: In case anyone was wondering about the additional issue I was mentioning, it was solved by clearing out the default texture on the RawImages and changing the Material to a new default Unlit/Texture shader.