search

w3c / encrypted-media

Encrypted Media Extensions
https://w3c.github.io/encrypted-media/
Other
180 stars 80 forks source link

MPEG-CENC Version Support #563

Open xhwang-chromium opened 2 weeks ago

xhwang-chromium commented 2 weeks ago

tl;dr: There are multiple versions of MEPG-CENC spec, and the EME spec is anchored on a specific version (v2, 2016). These versions are not always backward/forward compatible, i.e. a client supporting a newer/older does not automatically support the older/newer version. I'd like to discuss how EME should handle previous MPEG-CENC versions, as well as be able to handle future ones.

EME reference of MPEG-CENC

Currently the EME spec refers to the MPEG-CENC spec directly or indirectly in at least two places. And it's referencing a specific version: 23001-7:2016.

In 8.9.2 Interoperably Encrypted, we have

The Encrypted Media Extensions Stream Format Registry [EME-STREAM-REGISTRY] provides references to such stream formats.

where the EME-STREAM-REGISTRY specifies that

for MIME-type audio/mp4 or video/mp4, the byte stream format is derived-from an existing file format ISO Common Encryption ('cenc') Protection Scheme for ISO Base Media File Format Stream Format [EME-STREAM-MP4]

in which the MPEG-CENC spec was quoted as

[CENC] ISO/IEC 23001-7:2016, Information technology — MPEG systems technologies — Part 7: Common encryption in ISO Base Media File Format files. ISO/IEC. International Standard. URL: https://www.iso.org/obp/ui/#iso:std:iso-iec:23001:-7:ed-3:v1:en

In 3.4.1 Dictionary MediaKeySystemMediaCapability Members, on encryptionScheme, we specifies that

Well-known values for encryptionScheme are:

  • cenc: The "cenc" mode, defined in [CENC], section 4.2a. AES-CTR mode full sample and video NAL subsample encryption.
  • cbcs: The "cbcs" mode, defined in [CENC], section 4.2d. AES-CBC mode partial video NAL pattern encryption. For video, the spec allows various encryption patterns.

where the MPEG-CENC spec was also quoted as

[CENC] ISO/IEC 23001-7:2016, Information technology — MPEG systems technologies — Part 7: Common encryption in ISO Base Media File Format files. ISO/IEC. International Standard. URL: https://www.iso.org/obp/ui/#iso:std:iso-iec:23001:-7:ed-3:v1

MPEG-CENC Versions

Although EME spec only quotes the 23001-7:2016 version, there are multiple versions of MPEG-CENC existing. (There might be a v2 version but I am not familiar with it):

Encrypted Video Slice Headers

For NAL structured video, in CENCv1 video slice headers can be encrypted. And in CENCv3 and later, they must remain unencrypted:

Common Encryption specifies subsample encryption for NAL structured video that only encrypts video data, and leaves other NAL types, all NAL size and type headers, and video slice headers unencrypted.

The reality is that the are still a large number of existing content encrypted according to the CENCv1 spec, with encrypted video slice headers, especially for H264. This is normally not a problem for software decryptor/decoders. But for hardware ones, this could add significant complexity to the system because some hardware decoders require those slice headers to drive the decoding process. We have to find ways to decrypt the slice headers first and return the parameters to the decoder. Here's an example in Chromium that does this.

This adds a burden for the client implementations, and to the industry in general, that for every new implementation of hardware CDM (and decoder), they need to consider CENCv1 support, even though it's not referenced directly in the EME spec, to avoid breaking existing content. On ChromeOS, we decided not to support CENCv1 HEVC content to avoid the complexity, assuming there are less such contents since HEVC spec/content is newer. Nevertheless we still hit issues with playback failure because content providers have HEVC content with encrypted slice headers.

Newer and Future MPEG-CENC Versions

In 2023, there's a newer version of MPEG-CENC 23001-7:2023 that adds Content Sensitive Encryption scheme sve1 and support for multiple keys and IVs per protected sample, to name a few. (I am still learning what these features are about.) The EME spec doesn't have to be updated to support all new features in MPEG-CENC, but it would be great to clarify our stance on newer MPEG-CENC versions.

Goal and Suggestion

I hope the EME spec can clarify MPEG-CENC version support, such that:

One idea is to add a way in EME to detect MPEG-CENC version support. So that the application will not send CENCv1 content if the user agent says it's not supported. Or this can be in the form of feature detection (instead of MPEG-CENC version detection), such that in the future, the application can ask the user agent whether a new feature in a new MPEG-CENC version is supported.

cconcolato commented 4 days ago

Note that MPEG is currently discussing how to better signal Common Encryption features. See https://www.mpeg.org/wp-content/uploads/mpeg_meetings/146_Rennes/w23814.zip which starts with:

ISO/IEC 23001-7 relies on ISO/IEC 14496-12 structures (e.g., boxes) for signaling various aspects of encryption. As ISO/IEC 14496-12 evolves and adds versions/flags to boxes used by ISO/IEC 23001-7, the File Format group considers defining brand(s) for signaling the exact support required. A first attempt at describing this exact support is given in this document. The intent is to describe what is currently supported in the field. Additional brands may be defined in the future to cover recent tools that are not yet widely deployed or future tools. Experts’ feedback is encouraged either through GitHub or through MPEG contributions.

xhwang-chromium commented 3 days ago

@cconcolato Thank you for the info and the reference. I took a quick look and it does have some similarity to the problem we are trying to solve.

For EME, it seems we might need a more generic feature detection mechanism, other than the CENC version detection. For example, "encrypted-slice-header" could be one such feature.

Another example I can think of is the "Clear Lead" support, where the playback starts with clear buffers, and switch to encrypted buffers later. Most implementations support it well, but we do see some implementations not supporting it. And I do see different user agents trying to use different ways to signal the lack of support.