Role of CMAF

[vasilvv]

There is no actual description of how to use CMAF in the document, but the current text suggests that it will use CMAF.

The charter says "The proposed solution will provide extensibility for supporting different media formats and will support multiple media types and media encodings.", which to me suggests that we basically have to support alternatives to CMAF even if CMAF is the MTI thing we define at first. I suspect that if we define the boundary between what the protocol does and what CMAF does correctly, this will solve the problem by itself.

[suhasHere]

+1 to @vasilvv

I think it would be good to say what are the things we need to describe and then have subsections on how they map to different container formats.

May be start with CMAF and Raw QUIC to cover most of the use-cases.

[kixelated]

Yeah I added CMAF right before the draft deadline. In hind-sight it wasn't necessary and needs more specification.

[kixelated]

I need to actually read the full CMAF spec, but terminology for reference:

• CMAF Header: Headers contain information that includes information for initializing a track.
• CMAF Segment: A sequence of one or more consecutive fragments from the same track.
• CMAF Chunk: A chunk contains a sequential subset of samples from a fragment.
• CMAF Track File: A complete track in one ISO_BMFF file.

There's discussion (#21) on if the CMAF Header should be sent as a separate stream. I think "Warp Segments" are a superset of CMAF Chunks, since they don't need to be sequential (but do need to be ordered). We either need to change that, or mention that Warp is backwards compatible with CMAF, but not equivalent.

[fluffy]

Seems clear to me that we will need to allow CMAF to be used but I think we will also end up with things like audio only applications (thing clouhouse) where the overhead of CMAF would be very high and they will want to be able to use RTP style media types instead. I'd put a strong bet that when the dust settles we will need support for both CMAF and RTP style container and possibly extensibility for others. Probably not much to this other than some enum that indicates what container type is in the payloads.

[kixelated]

Yeah, a raw container does seem necessary at some point. At a minimum we need DTS and PTS for each frame.

Anecdotally, here's the statistics for 2 channels of AAC audio encoded at 128 Kb/s @ 44.1KHz and fragmented at "frame" boundaries (23ms):

init segment:   765 bytes
segment header: 24 bytes
frame header:   100 bytes
frame payload:  350-400 bytes

It's not as bad as I thought but it could definitely be improved. The fragmentation can be done at a configurable frequency to reduce the overhead but increase the latency.

[afrind]

Has this moved out of the base protocol and into the media-format spec or do we need to leave this open.

[vasilvv]

We can close this.