search

w3ctag / design-reviews

W3C specs and API reviews
Creative Commons Zero v1.0 Universal
332 stars 56 forks source link

WebTransport #389

Closed vasilvv closed 3 years ago

vasilvv commented 5 years ago

おはようTAG!

I'm requesting a TAG review of:

Further details:

You should also know that...

We'd prefer the TAG provide feedback as (please select one):

dbaron commented 5 years ago

A few very preliminary comments from spending an hour or two starting to look over the various documents:

From looking at the explainer, it seems like the "Detailed design discussion" and "Alternative designs considered" sections could perhaps be fleshed out a little bit more. One example of something I'd have liked to learn from those sections that I didn't would be how this proposal differs in scope from the previous RtcQuicTransport proposal; I think there are probably a bunch of other big picture things that are in your head, aren't obvious, and would be useful for the TAG to understand while reviewing.

It would likely also be useful for the explainer to explain how this spec relates to the WHATWG Streams specification.

It would also be useful to understand why the spec currently doesn't expose these in workers, and whether there's a future plan to do so.

Is there a section somewhere, either in the explainer or one of the two specs, that explains how the two specs (the one in WICG and the one in IETF) fit together?

A few very detailed comments on bits that I noticed while jumping around to look at some examples of things:

I'd note that the if (!!chunk) in one of the examples is a C++-ism and the !! is not idiomatic in JS (although not really needed in C++ either).

receiveDatagrams() may only be called once at a time. If a promised returned from a previous call is still unresolved, the user agent MUST return a new promise rejected with an InvalidStateError.

This seems a little bit odd, and likely to be a bit of a programming hazard (producing rejections in unexpected ways, depending on timing/races). Would it make more sense to return an empty sequence? That said... in the context of understanding why this method is asynchronous -- would it even be guaranteed to be empty? If it's asynchronous because the datagrams are on their way over from another thread or process... might asking for more not yield more?

Also, it seems like the type of receiveDatagrams is slightly wrong, since the prose says that it allows nulls in the result. It seems like it should be Promise<Sequence<Uint8Array?>>.

dbaron commented 5 years ago

(I realize those detailed comments may quickly become irrelevant if there's a big refactor to integrate with the streams spec, as one of the other links suggested.)

pthatcherg commented 5 years ago

Responses to dbaron's comments from 10 days ago:

RTCQuicTransport is the p2p impl of a WebTransport. I made https://github.com/WICG/web-transport/issues/41 to track making that more clear.

The explainer has now been updated to use WHATWG streams. We plan to update the spec to use them as well after some more discussion on how WHATWG streams should work (in particular for creating send streams and bidirectional streams).

We definitely want WebTransports to work in WebWorkers, we simply didn't think to say that explicitly.

The W3C spec references the IETF specs. Victor made https://github.com/WICG/web-transport/issues/37 to track that.

I can't find any example of "!!". Perhaps that went away when we moved to WHATWG streams in the explainer.

On the nullability of receiveDatagrams(): you're right, that's a mistake. But we decided on a different approach that doesn't return null (see https://github.com/w3c/webrtc-quic/issues/124). We just need a PR for it.

On the goofiness of receiveDatagrams() only being called once at a time: that will probably go away with the switch to WHATWG streams.

dbaron commented 5 years ago

@cynthia and I are currently looking at this in a breakout.

So far I'd note that we found that there are additional documents that seem critical to understanding this, in particular:

cynthia commented 5 years ago

Blunt and clueless question - what makes an Http3Transport unique over a QuicTransport (aside from the protocol being on another application protocol), and what service does this provide that fetch() is unable to?

We have a pile of other questions on this and the other two related specs - I can block out some time to get a full overview if you folks have time.

pthatcherg commented 5 years ago

Http3Transport allows at least 2 things over QuicTransport:

  1. Going through HTTP proxies or firewalls that only allow HTTP.

  2. Sharing a connection with HTTP traffic (to the same server with the same port rather than a different IP and port, and contained in the same congestion control context).

(Maybe Victor can remember more)

Vs fetch:

  1. The server can push streams down to the client

  2. The client and server can both send datagrams (unreliable, unordered).

Looking forward to a pile of questions :).

pthatcherg commented 5 years ago

By the way, the spec has been updated to use WHATWG streams everywhere (to resolve a previous question).

vasilvv commented 5 years ago

Http3Transport is currently the less thought-out part of the spec, as we are trying to better understand the use cases of people who want to use it. Its basic idea is along the lines of RFC 8441, in that it tries to provide a way to establish a bidirectional non-HTTP data channel within an existing HTTP connection. One of its main distinguishing features is that when the connection already exists, it allows to bypass a lot of overhead related to setting up a connection.

Both Peter and I are going to be in Google Tokyo on Friday. Everyone should also be at TPAC, especially the Wed-Fri half of it.

dbaron commented 5 years ago

@torgo and I are looking at this in a (second) breakout at the Tokyo face-to-face meeting.

One of the things we're talking about is the relationship between a bunch of different pieces of work that are ongoing.

If my understanding of what's going on is correct, it seems like WebSocketStream (see #394) is about designing a better (uses promises and streams, supports backpressure naturally) JavaScript API for the existing WebSocket protocol, and this work here is about designing a new network protocol (on top of a more modern substrate) and having a JavaScript API to go along with it. One question in our minds is what the path for developers to use the various things is; it seems like in so far as WebSocketStream and WebTransport's JavaScript APIs are doing the same things, they ought to be able to work the same way, so that it's as easy as possible for developers to migrate from WebSocketStream (which we understand to be a shorter-term project) to WebTransport. So, if this understanding is correct, we'd strongly encourage these JavaScript APIs to be developed in tandem.

That said, I'm not particularly confident of that understanding gleaned from across a bunch of different explainers and other documents, so we'd be interested to hear how it's wrong.

pthatcherg commented 5 years ago

We can discuss WebSocketStream vs. WebTransport more in person (higher bandwidth), but I did want to mention one thing that's fundamentally different: WebTransport has streams and datagrams (ReadableStream of bytes and ReadableStream of Uint8Array). WebSocketStream has messages (ReadableStream of Uint8Array). That makes WebSocketStream's WebSocketConnection look a lot like a WebTransport's DatagramTransport, except that for WebSockets, they are in order and reliable and for DatagramTransport, they are out of order and unreliable.

cynthia commented 5 years ago

I'm interested in setting aside some time to discuss these details. Are you meeting during TPAC separately or is this is happening as part of WebRTC?

vasilvv commented 5 years ago

We have a breakout session on Wednesday: https://w3c.github.io/tpac-breakouts/sessions.html#webtransport

We also intend to meet during the WICG meeting, though I believe the WICG agenda has not been finalized yet.

dbaron commented 4 years ago

It looks like we missed our opportunities to talk about this in person in Tokyo or at TPAC. @ylafon and I are cycling back to this in a breakout at the TAG's Cupertino meeting.

I'm curious if there's something written down that explains the relationships between these various specs at a high level, related to the brief discussion that started in https://github.com/w3ctag/design-reviews/issues/389#issuecomment-530694381. (It's also useful for understanding how and why the Javascript APIs for them should differ.)

I'm also curious if there's something that explains the handling of consent to communicate (along the lines of the discussion that started in https://github.com/w3ctag/design-reviews/issues/296#issuecomment-418572658 and continued a bit in #303 and #304). Is there an intent that Http3Transport and QuicTransport would differ substantially (e.g., with the former using CORS and the latter using ALPN as described in 17.3 Protocol Security?). That said, now that we're discussing it, we're not even sure we understand the design around QUIC and HTTP3 and what the relationship is (although I think my understanding is that HTTP3 is built on top of QUIC and other things could use QUIC directly... but I'm not sure exactly what they do and don't get if they do that).

dbaron commented 4 years ago

For what it's worth -- maybe a more general way to frame my previous comment is that perhaps it would be helpful if the explainer assumed a bit less knowledge about the space. It doesn't need to explain the web from first principles, but having a few more paragraphs to explain the basics of how this fits in to the things it builds on or connects to would probably be helpful.

plinss commented 4 years ago

dbaron to invite Martin Thomson to a future call

cynthia commented 4 years ago

@ylafon and I discussed during our Wellington F2F.

We think this proposal is extremely interesting, and will bring a whole new set of capabilities to the web. However, we also consider this a pretty powerful feature - and was wondering about the implications of the constructor magically connecting, hence not being able to be a promise. (which would allow gating it behind a permission) We also had some concerns about port scanning through this mechanism (e.g. through timing based testing, albeit it would only work on UDP ports for now) but it does feel like gating this behind a permission would be good enough as a mitigation.

cynthia commented 4 years ago

@plinss @dbaron ping on the Martin Thomson call issue noted above.

aboba commented 4 years ago

@cynthia The WebTransport constructor was modeled after the WebSockets constructor which also immediately establishes a connection.

To address the port-scanning issue we could prevent WebTransport from connecting to ports on the "bad ports" list.

cynthia commented 4 years ago

@aboba WebSockets is less powerful, and hence was never gated. This is a significantly more powerful API, so I think different considerations has to apply. As for the bad port blacklisting, those I believe are TCP ports; not sure if that blacklist would apply here.

vasilvv commented 4 years ago

What are the specific ways in which WebTransport is more powerful than WebSocket that are of security concern here?

annevk commented 4 years ago

https://github.com/WICG/web-transport/issues/110 seems somewhat relevant for the TAG review. (Also applies to some of the other class names I realized, e.g., QuicTransport.)

At least from Mozilla's review of this I don't think we found reason to permission-gate this and even "bad ports" might not be needed due to the specifics of the networking handshake. @martinthomson might recall this better. Having said that, I believe @ricea still plans on doing a stream-based version of WebSocket and it seems this should be more analogous to that than to WebSocket itself.

martinthomson commented 4 years ago

Yes, I think that we can avoid the port list and the permissions. Port scanning can be addressed by having the connection fail with a minimum timeout and no error codes that expose how it failed (maybe with some exceptions that we can carefully vet). After that, you leak no information until the server consents to communicate with this origin.

The big concerns are that we keep this and the new WebSocket stuff in sync so that they share what they need to (and that we don't do the same work twice). I'm assured that there is good coordination with Google between the teams, so that much is good.

aboba commented 4 years ago

@martinthomson Are you referring to WebSocketStream? There are some differences, but they were closed as "won't fix": https://github.com/WICG/web-transport/issues/50 https://github.com/WICG/web-transport/issues/51

annevk commented 4 years ago

In light of https://docs.google.com/document/d/1La1ehXw76HP6n1uUeks-WJGFgAnpX2tCjKts7QFJ57Y/edit those make sense I think, https://github.com/WICG/web-transport/issues/54 appears unaddressed still.

martinthomson commented 4 years ago

Not sure that "Peter talked to Adam" is sufficient for a decision, though I appreciate that this is a personal submission, so it does have a different standard.

Now sure how the WebSocketStream API is necessarily distinct based on the proposal.

ylafon commented 4 years ago

Sangwhan and just discussed this and we are fine with not gating with a permission to avoid port scanning as long as there is no way to infer what is happening by doing time-based analysis (connecting to a filtered port takes far more time than a blocked one or one responding with another protocol).

cynthia commented 4 years ago

@dbaron and I discussed this during a breakout today, and concluded that given the above port scanning mitigations are in place we are happy to see this proposal move forward. Thanks for bringing this to our attention, and we plan to close this after discussing with the group.

We'd like the group to actively engage in review with relevant experts while development progresses. (In particular, networking is one of our weak points, so we might have missed something important.)