Container

[kixelated]

We're missing a section on the container. A container is needed because the codec bitstream doesn't have timing information, nor is it specified how to support more fringe features like captions and DRM. I would advocate for using CMAF (fMP4) unless somebody wants to take on the work of standardizing a "raw" container.

To summarize the hierarchy:

• the codec bitstream is packaged media container
• the media container is split into layers
• each layer is sent over a QUIC stream

[kpugin]

I think we need to: 1) Have a way to signal what container is used 2) Agree on "default" container and I agree it should be CMAF

[suhasHere]

+1 on need to signal the container being used. I think we need to atleast CMAF and raw container/something ( to keep the packet sizes small for audio). But yes these can happen in sequence starting with CMAF first.

[vasilvv]

I agree that an ISO BMFF derivative is the most plausible answer here (if we use CMAF, we need to make sure we actually want everything in it as-is).

The other answers could be:

• M2TS. This has an advantage (at least from the ingestion perspective) of being already widely supported in the TV broadcast ecosystem. On the other hand, the framing overhead involved is fairly high, and I don't think the advantage of being already used is that good -- since we're splitting video and audio into different QUIC streams, some form of remuxing is required regardless.
• MoQ-specific bitstream framing. I believe that we should not do that, as that would result in us having to define codec bindings for every codec forever. Later versions might do things like defining lower-overhead format for audio codecs, but that's definitely a v2+ feature.

[fluffy]

+1 on CMAF but also I think I would be happy to take on the work of also supporting an "RTP Style" container which would be pretty much what you find in the payload of an RTP packet and use the media type from IANA to describe the format inside it. I'm thinking this will be trivial to do but perhaps I am missing what is hard about this.

[VMatrix1900]

+1 on RTP style container. We already have those IANA registry and RTP payload format as RFC. Maybe we can reuse them. Sounds like RTP over QUIC?

[kixelated]

In #43, I define a mimetype for each track, which contains the type/container/codec. The receiver can choose which track to play based on those properties. For example: video/mp4; codecs=avc1.64001e.

Is this all we need? And some text changes of course to remove the CMAF specifics.