RFC 8441 support (websocket over http/2)

[njsmith]

RFC 8441 was published a few months ago. It defines a standard way to tunnel a websocket through an HTTP/2 connection.

I think the main relevance for wsproto is that it uses a different handshake – instead of a GET with Upgrade: and a funky sha1-based handshake, it's a CONNECT with some special headers, and removes a bunch of the funky stuff:

   Implementations using this extended CONNECT to bootstrap WebSockets
   do not do the processing of the Sec-WebSocket-Key and Sec-WebSocket-
   Accept header fields of [RFC6455] as that functionality has been
   superseded by the :protocol pseudo-header field.

   The Origin [RFC6454], Sec-WebSocket-Version, Sec-WebSocket-Protocol,
   and Sec-WebSocket-Extensions header fields are used in the CONNECT
   request and response-header fields as defined in [RFC6455].  Note
   that HTTP/1 header field names were case insensitive, whereas HTTP/2
   requires they be encoded as lowercase.

   After successfully processing the opening handshake, the peers should
   proceed with the WebSocket Protocol [RFC6455] using the HTTP/2 stream
   from the CONNECT transaction as if it were the TCP connection
   referred to in [RFC6455].  The state of the WebSocket connection at
   this point is OPEN, as defined by [RFC6455], Section 4.1.

[Kriechi]

This will be supported by Firefox 65, to be released by end of January 2019: https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/65#Networking https://bugzilla.mozilla.org/show_bug.cgi?id=1434137

[pgjones]

Looks to be in chrome, but behind a flag https://www.chromestatus.com/feature/6251293127475200 https://bugs.chromium.org/p/chromium/issues/detail?id=801564

[pgjones]

I think to support this the handshake part needs to be separable from the websocket part, as only the handshake differs between HTTP/1.1 and HTTP/2. Then the WSConnection class can gain a http_version argument and either carry out a h11 or h2 handshake on the connection.

I think we should make the websocket part separate as well in order to deal with a shared h2 connection - with h11 the connection is taken over by the websocket, whereas with h2 only a stream is. In this case the handshake parsing is likely done by the h2 parser with only the stream data thereafter passing to wsproto. I propose calling the websocket part Connection.

I propose then the following classes exist,

class H11Handshake:
    def __init__(self, client): ...
    def send(self, event): ...
    def receive_data(self, data): ...
    def events(self): ...
    @property
    def connection(self): ... # Returns a Connection when established

class H2Handshake:
    # Same interface as H11Handshake

class Connection:
    def __init__(self, client, extensions): ...
    def send(self, event): ...
    def receive_data(self, data): ...
    def events(self): ...

Then the WSConnection is repurposed to tie this together on the assumption that the connection will be purely for websocket data,

class WSConnection:
    def __init__(self, client, http_version): ... # Will either use the H11 or H2 handshake
    def send(self, event): ...
    def receive_data(self, data): ...
    def events(self): ...

This should work without any additional changes for the default h11 case, but allow h2 cases as well.

@njsmith @Kriechi @mehaase What do you think?

[Kriechi]

Yes - this sounds like what if've been trying to talk about in https://github.com/python-hyper/wsproto/issues/27 already.

In mitmproxy we would only every use your new WSConnection, because our proxy stack already does all the handshake.

Just to make sure we are on the same page: The first byte ever received or sent by WSConnection is a WebSocket frame. WSConnection never sees, hears, or speaks anything other than WebSocket frames.

For WSConnection.__init__ we could either pass in the HTTP handshake, or "plain" arguments, meaning the user has to parse & prepare the values beforehand (if the user is not using our provided H11Handshake / H2Handshake helper tools)

[pgjones]

@Kriechi I've confused you a little. In my proposal WSConnection is unchanged. Instead the Connection class is what you describe, in that it only talks WebSocket frames, and nothing else.

I think all that is required for the Connection.__init__ is to know if it is a client or server and to know the agreed upon extensions - I think these are the plain arguments you are referring too.

There is an initial implementation here

[pgjones]

There is a proposed solution in #102, however #102 is a solution only if you wish to use wsproto to manage the entire connection, yet this somewhat defeats the purpose of using HTTP/2 (no multiplexing). The current master allows for HTTP/2 websockets as it makes the extension handshake available separately, however you have to manage the connection yourself.

I think #102 is probably the best idea with documentation explaining how to make use of the extension handshake part. Yet I'm unsure what the best API wsproto should provide is.

[Kriechi]

Thinking with my mitmproxy hat again, we want to handle the h2 connection ourself, and only pass off a single stream (and its events) for a possible WS connection.

[pgjones]

I think #109 will allow this to be closed as complete.