RealtimeStreaming's Unity3D camera server sample crashes in the Editor (Runtime Error!)

[danilogr]

Hello, First, I would like to thank the authors for sharing RealtimeStreaming. It is a great example on how to use Windows's SDK to efficiently encode and decode a video stream using C++/WinRT targeting Desktop and UWP.

I am currently trying to run the Unity3D example realtime-server-webcam from the editor. While it works and shows my webcam view, it instantaneously crashes when a client connects to it.

This is the error that I get:

I am using Unity 2018.4.2f1 on Windows 1903. I tried using the Debug and Release versions of the RealtimeStreaming DLL.

[danilogr]

After debugging Unity with VS2019, I learned that this is where the crash happens var hr = VideoServerWrapper.exWrite(this.serverHandle, data, (uint)data.Length);

By pre-attaching Visual Studio's debugger to Unity3D, the crash narrows down to this: Unhandled exception at 0x00007FF88D8AA839 in Unity.exe: Microsoft C++ exception: winrt::hresult_invalid_argument at memory location 0x00000000005FCF68. occurred

And this is the stack track:

Interestingly, RTDLL RealtimeStreamingWrite always returns S_OK (see RealtimeStreaming_DLL.cpp at line 540):

526     RTDLL RealtimeStreamingWrite(
527         _In_ UINT32 serverHandle,
528         __in_ecount(bufferSize) BYTE* pBuffer,
529         _In_ UINT32 bufferSize)
530     {
531         NULL_CHK(pBuffer);
532     
533         Log(Log_Level_Info, L"Plugin::RealtimeStreamingWrite()\n");
534     
535         array_view<byte const> bufferArrayView{ pBuffer, pBuffer + bufferSize };
536     
537         auto rtServer = GetModule<RealtimeServer>(serverHandle);
538         rtServer.WriteFrame(bufferSize, bufferArrayView);
539     
540         return S_OK;
541     }

Any leads on how to fix this?

[danilogr]

After checking the console, this is what I found:

RealtimeServer::WriteFrame() - Invalid buffer size w=1280 - h=720 
Exception thrown at 0x00007FF88D8AA839 (KernelBase.dll) in Unity.exe: WinRT originate error - 0x80070057 : 'The parameter is incorrect.'.
onecore\com\combase\winrt\error\restrictederror.cpp(1014)\combase.dll!00007FF88EAAB690: (caller: 00007FF88EAA60BC) ReturnHr(1) tid(414c) 8007007E The specified module could not be found.
Exception thrown at 0x00007FF88D8AA839 in Unity.exe: Microsoft C++ exception: winrt::hresult_invalid_argument at memory location 0x00000000005FC878.
Exception thrown at 0x00007FF88D8AA839 in Unity.exe: Microsoft C++ exception: [rethrow] at memory location 0x0000000000000000.
Exception thrown at 0x00007FF88D8AA839 in Unity.exe: Microsoft C++ exception: winrt::hresult_invalid_argument at memory location 0x00000000005FCF68.
Unhandled exception at 0x00007FF88D8AA839 in Unity.exe: Microsoft C++ exception: winrt::hresult_invalid_argument at memory location 0x00000000005FCF68.

Turns out that culprit is here in WebcamServerSource.cs:

56        webcam = new WebCamTexture((int)Server.OutputWidth, (int)Server.OutputHeight);
57        debugImg.texture = webcam;
58        debugImg.material.mainTexture = webcam;
59        webcam.Play();
60
61        UnityEngine.Debug.Log("Webcam Playing at " + webcam.width + " x " + webcam.height);
62
63        webcam_interop = new Color32[webcam.width * webcam.height];
64        frameBuffer = new byte[webcam.width * webcam.height * 4];

On line 56, we request a WebCamTexture with predefined width and height. We allocate our buffers (lines 63 and 64) with what the webcam gives us back. Turns out webcam.with/height is not always what we request. Thus, the server will fail as it is waiting for Server.OutputWidth x Server.OutputHeight. (See the first two lines of the console, 8294400 != 1280 *720*4)