search

w3c / webrtc-nv-use-cases

Use cases for WebRTC NV
https://w3c.github.io/webrtc-nv-use-cases/
Other
32 stars 13 forks source link

Is the DRM requirement in the Low latency Broadcast with Fanout use case satisfied by data channels? #86

Closed jan-ivar closed 1 year ago

jan-ivar commented 1 year ago

The N36 requirement in the § 3.2.2 Low latency Broadcast with Fanout use case reads: "Support for DRM (containerized media) or uncontainerized media."

What does "Support for" mean?

  1. One way to interpret this would be that this is calling out that functionality enhancements are needed around data channels, because of the native A/V stack's lack of DRM support.
  2. Another interpretation would be a direct support for DRM in the WebRTC A/V stack.

It might be good to clarify what the requirement for "support" actually is.

aboba commented 1 year ago

Existing APIs make it possible to send containerized media (with or without DRM) over the data channel, and render it with MSE. This has been implemented in use cases such as cloud gaming and peer-to-peer caching (e.g. Peer5). However, there are some "gotchas":

  1. Implementations often depend on "low latency MSE" behavior, which is not well documented or uniformly implemented.
  2. Worker support (e.g. DataChannel in Workers and MSEv2) is needed to move the receive pipeline off the main thread.

For Issue #1, there has been some interest in moving from MSE to WebCodecs. However, WebCodecs does not currently support DRM, so there are open WebCodecs issues relating to (non-containerized) DRM support, Issues 41 and 483. If these issues were to be resolved (questionable) then allowing WebCodecs to do encode/decode while utilizing WebRTC's RTP transport, as is being advocated in other recently proposed use cases, would be sufficient.

My take is that support for DRM in the WebRTC A/V stack is not a requirement, and that we should remove requirement N36 and substitute N13 (DataChannels in Workers).

aboba commented 1 year ago

I have submitted PR 90 to remove requirement N36, substituting N13 (DataChannel in Workers).

vitaly-castLabs commented 1 year ago

I'd like to point out that it's not necessary to use DataChannels for DRM, we (castLabs) use regular media tracks as they are better suited and allow "ultra low-latency scenarios" (https://www.w3.org/2023/07/18-webrtc-minutes.html). The key is to keep aux info like SPS, PPS, video slice headers intact and encrypt only macroblock data - this way the rest of the WebRTC pipeline won't even know the difference.

Aforementioned "low latency MSE" behavior is the issue indeed (especially with Edge/PlayReady), and "native" DRM support in WebRTC would make things a lot easier and efficient. In my opinion it makes much more sense than extracting data from the browser via Encoded Transform and then pushing it back to the exact same browser via MSE.

EME-for-WebCodecs would be usable too, but only if it can ensure the decrypted media can be rendered without leaving TEE (Trusted Execution Environment), i.e. decoding and rendering should be very tightly coupled.

All in all I don't see a good justification of replacing DRM with N13.

dontcallmedom-bot commented 1 year ago

This issue was mentioned in WEBRTCWG-2023-09-12 (Page 27)

dontcallmedom-bot commented 11 months ago

This issue was mentioned in WEBRTCWG-2023-12-05 (Page 19)