Clarify codec name for isTypeSupported/addSourceBuffer interop of FLAC-in-MP4

[wolenetz]

Support by MSE for FLAC in MP4 is becoming more commonly requested (FF has it supported as of 51, Chrome is looking into supporting it in M62 -- https://crbug.com/666000).

Inspecting RFC 6381, it is unclear to me on first read what the "codecs" string should contain to interoperably ask about support for FLAC-in-mp4 because the MP4RA codecs listing (http://www.mp4ra.org/codecs.html) doesn't explicitly list FLAC.

This issue is to track discussion and obtaining agreement on interop for which codec string to support for indicating FLAC-in-MP4 support.

@jyavenard, is it simply 'audio/mp4; codecs="flac"' or 'video/mp4; codecs="flac"' (possibly along with other codecs for the other codecs in the stream)?

[jyavenard]

is it simply 'audio/mp4; codecs="flac"' or 'video/mp4; codecs="flac"' (possibly along with other codecs for the other codecs in the stream)?

Yes.

[wolenetz]

I raised a concern about the RFC-6381 codecs parameter for mp4ra codecs being case-sensitive in https://github.com/xiph/flac/issues/38#issuecomment-318705625.

Snippet from there, pertinent to this MSE bug:

Why is the "fLaC" fourcc mixed-case? Does it have to be? Is it too late to change it? And if so, must we make an exception in the MSE ISOBMFF bytestream spec specifically allowing either "flac" or "fLaC" codecs parameter to identify iso-flac?

[jyavenard]

https://w3c.github.io/media-source/index.html#dom-mediasource-addsourcebuffer The MSE states that addSourceBuffer "2. if type contains a MIME type that is not supported or contains a MIME type that is not supported with the types specified for the other SourceBuffer "

A MIME type as per the HTML standard is defined as "A valid MIME type is a string that matches the media-type rule defined in section 3.1.1.1 "Media Type" of RFC 7231. In particular, a valid MIME type may include parameters. [RFC7231]" (https://mimesniff.spec.whatwg.org/#mime-type)

Section 3.1.1.1 defines a MIME type is made of a type and a subtype and a parameter: "The type, subtype, and parameter name tokens are case-insensitive. Parameter values might or might not be case-sensitive, depending on the semantics of the parameter name. The presence or absence of a parameter might be significant to the processing of a media-type, depending on its definition within the media type registry."

values are case insensitive. https://tools.ietf.org/html/rfc7231#section-3.1.1.1

IMHO, the xiph/flac bug is incorrect, and the name of the new MP4 box should have nothing to do with the name of the codec (after all we don't use avcC as the name of the h264 codec)

[jyavenard]

oh, I misread, it's the parameter name that is case insensitive, not the parameter value.

[davemevans]

The problem seems to be, as stated in https://github.com/xiph/flac/issues/38, the MSE ISO BMFF Byte Stream Format requires conformance of MIME type to RFC6381, which states that the codecs parameter must be a case-sensitive match of the sample entry descriptor fourcc (fLaC, in this case). RFC7231 does allow for the value of codecs to case-sensitive.

At this stage it doesn't seem unreasonable to change the codec, though this would, in the short term at least, lead to a problem of an existing browser implementation and production workflow, and another non-interoperable but spec-compliant browser implementation (though we may be able to work round these problems in JS). This is not a reason not to get it right, of course.

Adding an exemption in the MSE bytestream spec doesn't feel like a great idea either, although it would be easier.

after all we don't use avcC as the name of the h264 codec

avcC is the AVC Configuration box, which is a child of the avc[1-4] visual sample entry, and we do use avc[1-4] as the codec (eg video/mp4;codecs="avc3.64001F")

[wolenetz]

Adding exemption for this into the MSE bytestream spec seems the reasonable way forward in V2 scope (to support it as "flac" in ISO-BMFF bytestream.)

[ktmf01]

I was pointed here from https://github.com/xiph/flac/issues/38, what is the status of this? What action is expected/necessary from a FLAC point of view? Is this something that should be handled here?

I know a lot about FLAC but next-to-nothing of MP4, so please enlighten me.

[ktmf01]

@wolenetz As you are the last person who commented on this, do you know what the status is?

[Jamie0]

Are there any current plans to explicitly make reference to the case-sensitivity in the specification?

It's worth noting that in the Safari 17 beta (on iPadOS) the MediaSource implementation now only accepts the FOURCC codec string "fLaC" (https://bugs.webkit.org/show_bug.cgi?id=260491), yet most implementations/samples seem to use "flac" in lower case.