QUIC header format/demultiplexing

[larseggert]

[I am raising this for @csperkins; text below is from him.]

If I understand correctly, the first octet of the new QUIC header is either:

 +-+-+-+-+-+-+-+-+
 |1|   Type (7)  |  Long header packet
 +-+-+-+-+-+-+-+-+

or

 +-+-+-+-+-+-+-+-+
 |0|C|K| Type (5)|  Short header packet
 +-+-+-+-+-+-+-+-+

Given the assigned types, this gives number values for the first octet in the ranges 0-3, 32-35, 64-67, 96-99, or 128-136.

RFC 7983 talks about how to demultiplex protocols running on a single UDP port, based on the values of the first octet. The important part is the new text table in Section 7, which demultiplexes based on the value of the first octet of the packet as follows:

                   +----------------+
                   |        [0..3] -+--> forward to STUN
                   |                |
                   |      [16..19] -+--> forward to ZRTP
                   |                |
       packet -->  |      [20..63] -+--> forward to DTLS
                   |                |
                   |      [64..79] -+--> forward to TURN Channel
                   |                |
                   |    [128..191] -+--> forward to RTP/RTCP
                   +----------------+

The QUIC headers don’t fit cleanly into this. The result is that it’s not possible to run demultiplex QUIC and WebRTC running on the same UDP port, since WebRTC uses STUN, DTLS, TURN and RTP/RTCP. Demultiplexing these is probably desirable, since it would let you switch from web traffic to interactive WebRTC or very low latency streaming without de-doing the NAT bindings on a port.

The simplest fix to allow the protocols to be demultiplexed is to set the top two bits of the first octet of a QUIC packet to 1 and reduce the size of the type fields:

 +-+-+-+-+-+-+-+-+
 |1|1|1|Type (5) |  Long header packet
 +-+-+-+-+-+-+-+-+

 +-+-+-+-+-+-+-+-+
 |1|1|0|C|K|Type3|  Short header packet
 +-+-+-+-+-+-+-+-+

There are probably other options that keep a larger type field, if we accept a less obvious bit packing, although even these likely lose one bit from the type field.

[martinthomson]

I don't think that we need to multiplex QUIC with realtime media. (If we multiplexed every conceivable protocol, we'd run out of bits.)

[csperkins]

I tend to think the ability to do very low latency streaming on a UDP port you know works through NAT could be appealing.

Or, at least, shouldn't be accidentally prohibited...

[csperkins]

To be a little more general: we should decide whether we want to allow QUIC flows to be easily multiplexed along with some other UDP-based protocols on a single port.

Multiplexing with the full WebRTC stack is one example of this, but is not the only use case. For example, if we want to enable peer-to-peer QUIC in future, then we should consider defining the packet formats to be easily distinguishable from STUN and/or TURN to help NAT traversal.

[janaiyengar]

This may be one of those things where keeping WebRTC in the back of our minds might help QUIC not get stuck badly in the future.

So, as a thought exercise (not assuming that we need to solve this problem here) I'd like to walk through what we could do if we actually supported both WebRTC protocols and QUIC on the same port.

I think the simplest thing to do would be to pass everything through a QUIC "dispatcher" that determines if this packet belongs to QUIC --- basically confirms whether this packet should be consumed by QUIC or not --- and if it isn't consumed by QUIC (including packets that QUIC decides MUST be dropped because they are QUIC packets but violate some property) then it can be consumed by non-QUIC applications on the same port.

The question of course is how bad is the likelihood of false positives/negatives. With the encrypted packet types in QUIC, the QUIC dispatcher will safely reject non-QUIC packets.

With the plaintext types, which are used during handshake, QUIC packets are covered by a non-crypto FNV-1a hash. The cost that QUIC bears of a mis-classification among plaintext packets as a QUIC packet is limited to cases where the contents of the packet are not part of the final key derivation. These packet types are limited to a few long-form packet types, all of which have a number of version-independent fields upfront, including version, which could be checked for sanity (since connection ID and packet number can be arbitrary.) This still doesn't seem great, since 0x0001 is a fine version number to have in QUIC and may be a completely reasonable string in RTP. So alternatively, we could renumber the long-form packet types to start at 255 and count down, since 192 - 255 is left open by RFC 7983, and since we just need 6 cleartext packet types at the moment to be distinct from the RTP packet types (it's smaller than that, since Stateless Reject and Version Negotiation packets need to echo stuff from the client's packets.)

TL;DR of this is that if we wanted to allow for a future coexistence of WebRTC protocols with QUIC on the same port -- and this is a big "if", since I wonder if they need to be on the same port --- we could renumber just the long-form packets to start at 255 and grow downwards. We only need <= 6 plaintext packet types to be out of the collision region with RFC 7983.

[mnot]

Discussed in Paris: We don't believe we need this. If we determine that it is needed later, we can add a framing layer that would be negotiated in a signalling channel.

[janaiyengar]

We're clearly not done with this discussion yet, per discussion in Singapore and #956. Re-opening this issue so that discussion happens on the issue and not on the PR.

[ianswett]

I have a few open questions and comments about this before we resolve it.

1) What are we trying to avoid overlapping with? My previous understanding was that we only cared about STUN, since we could deal with the others overlapping. Also, doesn't STUN have some large magic cookie to make it easy to identify when multiplexing? 2) Particularly given the cookie, why is Martin's heuristic solution in https://tools.ietf.org/html/draft-aboba-avtcore-quic-multiplexing-01#section-2.3 insufficient? 3) Why do we need to flip the connection id bit to be omit connection id? That removes flexibility for implementations supporting both elsewhere and might make greasing harder. 4) If we need two bits for spin bits or spin+loss(or just reserving them), as is being discussed, we can't put two adjacent bits anywhere and still use the nonce bit to distinguish IETF QUIC from GQUIC, because it will no longer always be 1, hence question 3. I suspect this might make @britram and others a bit unhappy?

[martinthomson]

Can we try not to lump all the issues together? We don't have greasing yet and nor do we have a spin bit. Though #956 was an attempt to avoid collisions, nor does it guarantee avoidance in future. We can assess the impact on use with 7983, for which I recommend following the guidance here. In other words, we aren't obligated to continue to make it work, though we might try.

That leaves the heuristics (which includes the STUN cookie). As discussed, those can be used, but are inefficient and awkward.

[janaiyengar]

Thank you for your draft on muxing principles -- I think that should have been part of the discussion here prior to committing #956.

I agree that we don't need to design for things that haven't been agreed upon yet (spin bit, greasing), but it's a conversation that may (yet again) have us changing packet types. I don't see harm in anticipating it, but I agree that we shouldn't hold up progress in anticipation of uncertain issues.

That said, AFAICT, we have not agreed on creating a scheme that muxes QUIC with everything in RFC 7983. From the interim, I took away that STUN was probably important, but we didn't care much about the rest of the protocols. That was what I heard @csperkins ask for, and it is what I agreed with -- that maybe we should allow muxing with STUN for now, and I had argued that we leave the rest out. I'm still of that opinion. Practically, if we're only muxing with STUN, we could use a different solution space altogether (basically reserve those 4 types and we're done), but we seem to be trying to accomodate all protocols in 7983. I'd like to see agreement on that.

And if we're only doing STUN, then I've missed the discussion on inefficiencies in using the STUN cookie -- it's not on this bug, where is that discussion?

[marten-seemann]

If we only care about STUN, we could revert the inversion of the connection ID flag in #956. This would make the transition from gQUIC to IETF QUIC easy, and it would give us freedom how we use the remaining short header bits (as long as we make sure that 0x0 - 0x3 are reserved for STUN), be it for measurability bits or for greasing.

[ianswett]

I think it's worth considering all the relevant issues when making this change, because at some point in the near future, we need to stop changing the type byte for long enough to deploy some production code on the internet.

As Marten said, switching back to "connection ID" from "omit connection ID" would leave us in a place where we can do anything we want with the remaining 6 bits, including greasing and/or spin and still be able to coexist easily with GQUIC, so I propose we do that unless there's a strong argument against it?

[janaiyengar]

Copying discussion from #956 so that it's happening on this bug instead.

@ronieven said: "My understanding from the meeting was that there is no need to multiplex short header with stun so the change of the type values for the short header is a surprise and I did not see any discussion on the QUIC mailing list about this change. It is not mentioned also in the Singapore meeting notes. The change that was agreed as far as I remember was to flip the value of the connection ID flag since it will be on for the peer to peer case."

@csperkins said: "We certainly need to demux QUIC short header packets with STUN, to maintain bindings.

Changing the packet type values doesn't prevent the spin bit, although it might change which bit is used as the spin bit."

@britram said: "Note also that #989 would give us more degrees of freedom in the short header first octet."

@ekr said: "Colin is right here. You have to multiplex short headers with STUN b/c of consent checks, as well as candidates you learn about late"

[janaiyengar]

@csperkins: I appreciate the need to mux with STUN packets. RFC 5389 says however: "In some usages, STUN must be multiplexed with other protocols (e.g., [MMUSIC-ICE], [SIP-OUTBOUND]). In these usages, there must be a way to inspect a packet and determine if it is a STUN packet or not. STUN provides three fields in the STUN header with fixed values that can be used for this purpose. If this is not sufficient, then STUN packets can also contain a FINGERPRINT value, which can further be used to distinguish the packets." Isn't this FINGERPRINT to be used for exactly this sort of use case where it conflicts with other packet types? Why doesn't demuxing on the FINGERPRINT work?

Separately, I'm not convinced that we need to mux with other protocols. If we agree on the need to mux QUIC packet types with STUN, then we lose only 4 packet types... but we seem to have put aside a lot more than that in #956.

[csperkins]

If all you care about is demuxing QUIC and STUN, then there are several alternatives. The fingerprint is one option, certainly. Inspecting the first two bits of the packet and checking for the presence of STUN Magic Cookie is another that's simpler (especially if we revert the C bit in the short header packets).

I do think there are use cases for demuxing QUIC with other protocols, however (@aboba likely has input too). #956 (and the -08 draft) supports this with only minimal changes to QUIC (renumbering fields, no semantic changes).

We do need to document a long-term stable demux algorithm for QUIC and STUN, and decide to what extent we want a stable algorithm for demuxing QUIC and other protocols. My feeling is that one of the QUIC invariants has to be an algorithm for demuxing QUIC and STUN, but for demultiplexing QUIC and other protocols we should say "this works for QUIC v1, but we don't guarantee for future versions of QUIC".

[martinthomson]

@janaiyengar, we have reserved no packet types. We have just changed the values we use. If we don't have the freedom to do that between QUIC versions, then the entire invariants exercise was pointless. As @csperkins says, the hacky multiplexing scheme used for realtime uses will need to assess every QUIC version independently. That's possible viable there because of the use of session negotiation (SDP and all that).

[ianswett]

I completely agree with your last point Martin, that we have no reserved packet types. However, we still have some important issues to resolve, including greasing and possibly changing the packet number format, and we should resolve those before we do hacky changes, since solving those will likely cause us to do different hacky changes in the future. Using TLS 1.3 as a model, I think we should hold off on hacky changes until towards the very end of the process.

I sent out PR #995 because draft 08 already introduced a ton of changes, this one was unnecessary at this stage, and there is a lot of discussion on this matter still, so there's no point in changing things now. I'm happy that we're having that discussion on the list and here, so I think we'll figure out what we really need to make QUIC and WebRTC coexist.

I'd also like to be in a place where we can QUIC can coexist with other protocols fairly easily in general. I don't have any concrete proposals yet, but UDP 443 is fairly heavily used, so it'd be nice to find a more general solution.

[janaiyengar]

@csperkins: Agreed that #956 changes packet type syntax only, and it solves the problem of avoiding any collision with the space in RFC 7983. But as @ianswett points out, we want to do greasing, which might introduce semantics into this space. From where I see it, we can't be done sorting out packet types until two things are resolved: which collisions to avoid, and how, if possible, to do greasing.

@martinthomson: I agree that we don't have any reserved packet types, and so we are free to change them across versions. My note earlier was about this version of QUIC -- QUIC v1, or more precisely, draft -08. The packet types chosen in #956 have a very specific end-goal -- to avoid a collision with 7983. Given that, only one of two things is true: either QUICv1 packet types avoid a collision with 7983, in which case we are setting those types aside as not usable (what I termed "reserved") for QUIC v1; or QUICv1 packet types don't avoid a collision with 7983, in which case #956 is entirely cosmetic. Which of these is reflected in the packet type choices in draft -08?

[csperkins]

@janaiyengar is greasing a problem? Grease as much as you like, unless you want to multiplex. If you want to multiplex, you know what you're multiplexing with, and so can avoid those packet types when greasing. Yes, we should write this down.

[janaiyengar]

@csperkins I agree, and that's exactly the discussion I'm trying to have -- specifically, what do we need to multiplex with?

[csperkins]

@janaiyengar we need to multiplex with STUN, else we'll end up reinventing it within QUIC when we want to run peer-to-peer.

We want to multiplex with WebRTC (DTLS, RTP, STUN) since people want to use QUIC in place of the WebRTC data channel (and because it might be useful to run WebRTC from the server you're running HTTP/QUIC from). We likely also want to multiplex with gQUIC as a transition.

In practice, I think we need to mandate STUN demux in the QUIC spec. Anything else can be left for a separate document – it doesn't affect how QUIC greases, and muxing other protocols doesn't have to be written into the QUIC documents. We just need to be a little careful as we define QUIC v1.

[janaiyengar]

I agree with most of your classification, with the caveat that gQUIC is a slightly different beast in that we need to be able to demux for a short while until everything is off of gQUIC, and then we don't need to worry about it anymore.

On STUN, I heard you say earlier that we can use the FINGERPRINT for this purpose. Is it reasonable to use this instead of the packet type to demux from QUIC?

[ronieven]

If we only want STUN demux we had a problem with the short header and my recollection is that the solution was to flip the value of the C bit since connection ID will not be needed in peer to peer assuming that for client server we will not need STUN.

[csperkins]

@janaiyengar you can't just use the fingerprint. The algorithm to identify STUN is to check that the first two bits of the packet are 00, then check the magic cookie (bits 33-64 = 0x2112A442), then if necessary find and check the fingerprint attribute. Checking the fingerprint is relatively expensive.

QUIC long header packets don't conflict with STUN, since QUIC long header packets start 0b1x and STUN packets start 0b00. The fingerprint isn't needed.

QUIC short header packets don't conflict if either the C bit is 1 or the C bit is set to 0 and bits 33-64 differ from 0x2112A442. We can avoid collisions without requiring the STUN FINGERPRINT by making the C bit an Omit Connection ID bit (as in #956), and changing the Connection ID to be either (a) a 63-bit value with bit 64 always set to 1; or (b) a 56-bit value in the range 64 to 2^62-1 (i.e., a 64-bit value where the top two and the lower six bits are set to zero) formatted using the variable length integer encoding.

Both these changes seem conceptually clean, and would be QUIC version independent and fit with the invariants.

That said, I think there's value in demuxing with WebRTC traffic in addition to STUN, at least for QUIC version 1. Renumbering the fields as in #956 is independent of the above discussion of how to demux STUN, doesn't change QUIC semantics, and enables us to do this.

[marten-seemann]

@csperkins: Does your argument assume that connection IDs are never omitted when QUIC is multiplexed with STUN? If so, we have a tradeoff here (is sending the 8 byte connection ID on every packet more expensive than checking the fingerprint?).

[csperkins]

@marten-seemann no, it doesn't assume that. In short header packets C = 1 to mean connection ID is omitted avoids collisions with STUN. If the Connection ID is present, then C = 0 which makes the first two bits of the packet match those of STUN packets, but if we change the legal values of Connection ID as in my previous we still avoid collisions with STUN when the Connection ID is present. No STUN FINGERPRINT needed in either case.

[ianswett]

@marten-seemann no, it doesn't assume that. In short header packets C = 1 to mean connection ID is omitted avoids collisions with STUN. If the Connection ID is present, then C = 0 which makes the first two bits of the packet match those of STUN packets, but if we change the legal values of Connection ID as in my previous we still avoid collisions with STUN when the Connection ID is present. No STUN FINGERPRINT needed in either case.

Sorry, but I'm still a bit confused on the connection ID bit flip in general.

As Marten mentions, there are use cases for both omitting the connection ID from all packets when multiplexing QUIC with WebRTC and use cases for including it. So inverting the meaning doesn't help solve any problems, unless you don't care about one of these?

[csperkins]

I was misreading the packet formats, which perhaps wasn't helping... But, to summarise, a STUN packet header is formatted as:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0|     STUN Message Type     |         Message Length        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                   Magic Cookie  = 0x2112A442                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
|                     Transaction ID (96 bits)                  |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

This is straight-forward to demux with QUIC long header packets, since they have the initial bit set to 1 and STUN has it set to zero.

QUIC short header packets are:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0|C|K|   Type  |     [Connection ID (64)]                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     [Connection ID (64)]                      |
+               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|               |X X X X X X X X X X X X X X X X X X X X X X X X 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

I assume the common case is that the Connection ID is not present. If we define C=1 to mean that the Connection ID is not present, then the first two bits of the QUIC short header packet will be 01. This is straight-forward to demux from STUN, since those bits are 00 in STUN packets.

If the Connection ID is present, then the QUIC short header packet will have C=0 and so start with the first two bits equal to 00, which matches the first two bits of a STUN packet. We also see that the 64-bit QUIC Connection ID occupies the same space as the last 8 bits of the STUN Message Type, the 16-bit Message Length, the Magic Cookie, and the first 8-bits of the Transaction ID. To demultiplex such a QUIC packet from STUN, we need to either (1) use the STUN FINGERPRINT; (2) redefine the QUIC Connection ID to be 63-bit value that encoded into a 64 bits for both long- and short-header packets, with the encoding chosen so that one of the middle bits is set to avoid collisions with the STUN magic cookie when sent in short header packets; or (3) choose the QUIC short header type values to avoid conflicts with STUN message types (which is what #956 does). None of these are pleasant, but I tend to think (3) is best, since it also allows demux of WebRTC traffic.

[janaiyengar]

@csperkins : Does this mean that the short header type values in #956 are adequate and that redefining the C bit is unnecessary? That seems to follow unless I'm missing something.

I expect that the Connection ID will be present on all packets going from the client to the server (including ack packets) and that it will mostly be absent on packets going from the server to the client. This is what GQUIC does at the moment, and Google servers will require the client to always include connection ID on packets it sends.

[martinthomson]

Discussed this in Melbourne and the group there decided to close this issue with no action. The current state of the documents reflects the direction we decided to take. The chairs will decide on when and how to assess formal consensus.