[BUG] Worse performance when using the Receiver scene compared to a web browser

[ZeoWorks]

Hello again, It appears the stream is more intensive on the GPU when using the "receiver scene" as opposed to using a web browser (webserver video index). Furthermore, the stream doesn't stay at a consistent 60fps within the "receiver scene". It seems that streaming a video straight to Unity through webrtc has worse performance overall compared to the web browser. Latency is worse, it's more expensive on the GPU and you'll find micro stutters from time to time.

Thanks.

[karasusan]

@ZeoWorks Thanks for your feedback. This might be the same issue and the fix is already planned near future. https://github.com/Unity-Technologies/com.unity.webrtc/issues/205

[ZeoWorks]

@karasusan Thanks, This actually occurs with software encoding as webRTC for windows standalone builds does not support hardware decoding.

[ZeoWorks]

@karasusan More debugging done; There is an additional 50ms/90ms of latency when software decoding inside a Unity standalone/editor compared to software decoding within a web browser.

[karasusan]

@ZeoWorks Thanks for sharing details. We will focus to improve performance for the next WebRTC version.

I would like to know more detail.

• Testing platform (Win/macOS)
• Testing codecs (VP8 should be selected in default)

[ZeoWorks]

Testing platform; Windows to Windows, Codec; VP8 (Also occurs with VP9), Latency in 'Receiver scene' VS web browser; -50/90ms (sometimes upwards of 100ms) -compared to 29ms in web browser FPS in 'Receiver scene' VS web browser; -35-48fps -compared to 60fps in web browser

[karasusan]

memo: URS-348 memo: WRS-165

[doctorpangloss]

There is an additional 50ms/90ms of latency when software decoding inside a Unity standalone/editor compared to software decoding within a web browser.

I would add in general, compared to Moonlight, com.unity.webrtc has worse latency, closer to 40ms difference. Not 100% sure why. libwebrtc shouldn't be that bad to Chrome.

[karasusan]

@doctorpangloss

Looks like they implements libraries based on NVIDIA GameStream. https://github.com/moonlight-stream/moonlight-common-c

NVIDIA GameStream https://www.nvidia.com/en-us/shield/support/shield-tv/gamestream/

[ZeoWorks]

Same principle as using the nvidia SDK actually. The main reason is that the SDK is not in "high performance low latency" mode.

But yes, the latency of software decoding inside Unity (standalone) is much higher than in web browser regardless.

[karasusan]

@ZeoWorks @doctorpangloss I had experiment to check the latency as you said and reproduced your issue.

Unity Editor : 2020.3.23f1 on Windows Google Chrome: M96 Video Codec: VP8

• Win Unity Editor -> Chrome M96: 83ms
• Win Unity Editor -> Win Runtime: 149ms

It is clear that the video receiver implementation has problem of latency. First I will add statistics viewer like webrtc-internal to measure the latency more detail.

[ZeoWorks]

@ZeoWorks @doctorpangloss I had experiment to check the latency as you said and reproduced your issue.

Unity Editor : 2020.3.23f1 on Windows Google Chrome: M96 Video Codec: VP8

• Win Unity Editor -> Chrome M96: 83ms
• Win Unity Editor -> Win Runtime: 149ms

It is clear that the video receiver implementation has problem of latency. First I will add statistics viewer like webrtc-internal to measure the latency more detail.

Thank you for the update in regards to the situation. :) Best of luck sir, happy to help any way I can.

[karasusan]

I made the tool to measure latency between sender and receiver. https://github.com/Unity-Technologies/com.unity.webrtc/pull/586

We need to concern that the condition affecting results are FPS and video resolution.

[karasusan]

@ZeoWorks I measured latency using this tool, there was no problem. We will check the stats to measure performance for encoder/decoder.

[karasusan]

memo: WRS-205