Are QUIC streams the right choice

[goelvidhi]

I am not an expert in media inject / distribution protocols but I would think that QUIC stream is not the right design choice for anything that is time sensitive as the data is not delivered to the receiver (anything above QUIC) until it is complete and ordered. Is that not the case for the use cases defined in this charter? The mentioned use cases include gaming, streaming and web conferencing and I think that these use-cases require delivery of most recent media packets even if all are not received.

[afrind]

I think the idea is to only put data on a given stream if that data has a strict ordering dependency anyways. Also note that while gaming and web conferencing are use cases in the media-over-quic space, the initial charter is considering limiting the scope of work to live media ingest and distribution which does not have the same strict latency requirements.

QUIC streams are a very useful building block that handle the ordering, loss detection and recovery automatically so shouldn't be overlooked as a tool for building powerful yet simple solutions.

[fluffy]

@goelvidhi - How to map media to the QUIC is definitely one of things that would need lots of thought and discussion once a WG was formed. There are a bunch of tools that come into play. At the far end of the spectrum, QUIC Datagram provided full out of order delivery. That might be the right thing for some cases but for many cases, the media is set up such that some the video P frames can't be decoded until the I frame has been received. That might mean that putting an I frame and all the P frames that depend on it into a single stream might be a good design. Most modern systems don't display partial video frames or frames with errors, so that might lead to just put each video frame in it's own stream. There are bunch of options with pro's and cons and I think we need a flexible solution.

[acbegen]

Timely delivery is a function of time (delivery rate gets (close) to 100% over time). I believe the charter for MOQ should consider a protocol that can achieve the best delivery rate given a (tunable) delay budget. On one extreme, we have plain UDP, and on the other extreme, we have TCP/fully reliable QUIC. The intermediate trade-off points are only attainable using a partially reliable protocol, which QUIC may offer.

[maxsharabayko]

@afrind:

I think the idea is to only put data on a given stream if that data has a strict ordering dependency anyways.

It makes sense for a media transport to group packets so that if preceding ones from the group are lost, the transport knows it does not make sense to insist on delivering the rest. For example, if you lose something from the slice NALU, the entropy decoder wouldn't be able to decode the rest because it loses the context. Thus it is reasonable to either recover the preceding loss or drop the whole group.

It also means the media transport protocol would need to multiplex elementary video and audio streams on its own, to put those into different independent groups. If someone would like to transmit, say, the MPEG-2 TS stream widely used in the broadcasting industry and supported by RTP, the grouping and prioritization become way trickier.

(..) the video P frames can't be decoded until the I frame has been received. That might mean that putting an I frame and all the P frames that depend on it into a single stream might be a good design.

There are different ways for a decoder to improve resiliency to missing portions of frames, and to show at least something even with errors. There are also different types of coding based on slice partitioning and GOP structure, where not only an I frame is the one being referenced, but also P and even B frames. GOP structures can be quite complex, and hierarchical. GOP can also be open, meaning it can reference frames from previous GOPs. This way all the frames in the video would be pulled into one QUIC stream, which does not make much sense.

I would suggest a media transport protocol be as agnostic to the content as possible. It might be aware of some priorities within the multiplexed stream, groups of packets, but I don't think it should make decisions knowing some packet is a part of an I frame, and another one is a B-frame, etc.

[LPardue]

I would rephrase this question to be "do we need to solutionize in the charter"? And my answer to that question would be no.

QUIC has two standard mechanisms to carry application data - streams and datagrams. Supporting either of these in a transport implementation is a solved problem.

The charter text says:

Media will be mapped onto QUIC streams as part of the baseline protocol and an extension mechanism will be defined to map the media onto QUIC datagrams.

How applications map their data to streams or datagrams is an open question, and something that a MoQ can work through if chartered. Mentioning any constraint for one or the other just seems like a distraction to me. Maybe others disagree. If they do it would be good to know why the poorly defined "baseline protocol" feels the need to do so.

[fluffy]

There are some people that think we should only haver streams. There are some people that think 20 year of VoIP has made it pretty clear why you need Datagrams. I think the reason this should be in the charter is that it much easier to get agreement on it now than argue about it in the WG for 2 years. If we can come to consensus quickly on this in forming the charter that is awesome. The current answer in the charter is basically do streams first, and you have to do stream, but we have an optional extension for datagrams for the people that want that. That also sets up the charter so the streams works happens first and datagram second.

[kixelated]

Streams are the way to go, but it feels out of place to put this constraint in the charter.

[LPardue]

Ahh, from that explanation, I see now that baseline protocol == "Media Publication Protocol". Can we tidy up the terminology regardless of the techincal outcome of this issue?

[aboba]

I don't think this is the right question to ask for a charter discussion. I have seen reliable/unordered transport (no head of line blocking) used effectively in low latency streaming as well as unreliable/unordered transport (datagrams), with very similar media encapsulations. It is quite possible to allow the application to decide how frames are mapped to streams or datagrams without prejudging this in the charter.

[huitema]

Designs like "one QUIC stream per frame" come close to using datagrams, but not quite. This is very visible when using a series of relays to distribute real time video to large groups. When the frame is many packets (e.g. I-Frames) then one can see head-of-line blocking on the stream itself, and then cumulative jitter accumulating in media relays. If you are fragmenting I-Frames over datagram, you can do reassembly end-to-end, and thus avoid head-of-line blocking. So there are definitely cases where datagrams provide significant advantages.

[aboba]

Is there a reason to prevent the application from deciding the mapping of frames to streams/datagrams? What if the application wants to send the I-frame over datagrams and P-frames via message/stream? Or to send individual layers (e.g. base layer) within their own stream?

[huitema]

@aboba there are all kinds of architectures that could be conceived, but it seems that we would be better off going for just two: one based on datagrams optimized for performance, and one based on streams for compatibility with H3 -- and possibly H2. But of course we will most likely explore all possibilities before getting there.

[maxsharabayko]

Cutting off QUIC Datagrams disables a lot of workflows, as well as cutting off QUIC streams. Including both in the charter makes much more sense to me, especially, to provide pros and cons, and a guideline or requirement on when to use each of those.

[SpencerDawkins]

There are some people that think we should only haver streams. There are some people that think 20 year of VoIP has made it pretty clear why you need Datagrams. I think the reason this should be in the charter is that it much easier to get agreement on it now than argue about it in the WG for 2 years. If we can come to consensus quickly on this in forming the charter that is awesome. The current answer in the charter is basically do streams first, and you have to do stream, but we have an optional extension for datagrams for the people that want that. That also sets up the charter so the streams works happens first and datagram second.

I am sympathetic to @fluffy about wanting to have a discussion at charter time, so the lucky working group chairs don't have to make a consensus call after the working group argues for without coming to agreement.

Having said that ... if there are some people who can't live with streams-only - and I suspect there are also people who can't live with datagrams-only - that might because there are two groups who are solving different problems in the same BOF, and it's reasonable for each of the groups to be in MOQ because ... they are both transferring media ... over QUIC ... and MOQ is the only BOF in town.

Expecting either of the groups to wait until the working group solves the problem the other group has, seems like a bad idea. If there are multiple groups, one option would be to propose a charter per group.

[suhasHere]

closing the issue based on #34 and conclusions from the charter sync to keep decisions addressed... If there are specific aspects of the meta-issue that needs to be discussed, we should always open a new issue to address that matter.

[suhasHere]

@SpencerDawkins from what i recall from the charter sync, I thought the group was not split between streams-only or datagram-only, but rather "ok with having both" since it helped meet the various use-cases/applications. [ That's my read of the discussions on 4/22 sync though]

[murillo128]

and MOQ is the only BOF in town.

fwiw, RTP over QUIC (datagrams) is being done in AVTCORE

[aboba]

@murillo128 There is work on extending RTP over QUIC to message-based transports (e.g. frame per stream).

[murillo128]

what i mean is that there are going to be other ietf working groups doing media over quic. If we already have RTP over QUIC on AVTVORE using datagrams maybe it makes easier for MOQ charter to focus on a QUIC stream solution instead

[suhasHere]

I personally don't see the relation between support RTP over QUIC (via Datagrams or may be QUIC Streams) and another media protocol over QUIC (with 2 modes of operation) . These can be done in parallel.

Also its been happened many a times that we start with one mode and eventually we think of adding other modes. for example, QUIC started with stream only mode and then datagram support for added to enable use-cases with different set of requirements.

my 2 cents is we shouldn't block either efforts (RTP over QUIC, Media Over QUIC) . Nice thing is we can learn from each too.