search

crossbario / autobahn-python

WebSocket and WAMP in Python for Twisted and asyncio
https://crossbar.io/autobahn
MIT License
2.48k stars 766 forks source link

Reverse protocol handling for WebSocketResource #641

Closed warner closed 8 years ago

warner commented 8 years ago

I have a set of unit tests (in https://github.com/warner/magic-wormhole) that recently started failing when I combine autobahn-0.13.1 and Twisted trunk. It looks like https://github.com/twisted/twisted/commit/4f9e36d8ad21f9d73369f4d506bab30f61c2ad71 (landed 29-Mar-2016) is where the problem started (that's the first revision after Twisted-16.1.1 was branched off that causes my project's tests to fail).

The commit says:

Merge hide-request-transport-8191-5: Hide the transport backing the HTTPChannel object from twisted.web Resource objects.

And my tests are using WebSocketResource to put a websocket on a specific URL. I haven't tracked down the specific failure yet, but it seems like the server is having an internal error when the client attempts to connect to it, or the server isn't accepting the connection at all.

The relevant Twisted bug is at https://twistedmatrix.com/trac/ticket/8191 .

When I get some time (maybe next week), I'll figure out how to run autobahn's test suite against various versions of Twisted and see what happens.

Lukasa commented 8 years ago

Twisted doesn't support HTTP/2 plaintext upgrade (that is, upgrade using the Upgrade header). This is mostly for technical reasons: as you know, right now there are no good APIs for doing plaintext upgrade. This oversight is acceptable because no browsers support it either.

Twisted and the browsers support negotiating HTTP/2 using ALPN. This is a TLS extension which makes the negotiation happen during the handshake. That means negotiating HTTP/2 is no slower than negotiating HTTPS. That means the Time To First Byte is the same, but HTTP/2 imposes a minor (usually 24-byte) overhead for the initial SETTINGS frame exchange. This doesn't add much latency (it's asynchronous), and is the only difference between HTTP/2 and HTTP/1.1 over TLS. Given HTTP/2 header compression, the first request ends up taking roughly the same time for both, at which point HTTP/2's various efficiency improvements cause latency to be much lower for H2 that HTTP/1.1.

oberstet commented 8 years ago

I only care about latency until WebSocket takes over. I don't care about header compression or general latency of HTTP request (because we don't use that other than for serving a handful SPA assets). HTTP2 will establish at least one HTTP2 stream before WebSocket kicks in?

Anyway: this is all moot from Crossbar.io's perspective, because we want RawSocket, and the latency when Autobahn is in charge will be lower - I don't really have to measure that, since I know the code of Autobahn, and cycles spent will be less - the "Upgrade" to RawSocket using the MultiSocket POC I wrote above is absolutely minimal. We do care about WebSocket/RawSocket handshakes/sec as one performance metric.

For Crossbar.io at least, we won't use WebSocketResource anymore beginning with the next release. Of course I can't speak for other users of Autobahn using it at the plain WebSocket level .. they might want to continue using WebSocketResource.

I would be interested in a ALPN based negotiation mechanism for WebSocket though -- but that isn't spec'ed as far as I know=(

Lukasa commented 8 years ago

HTTP2 will establish at least one HTTP2 stream before WebSocket kicks in?

Not as far as I know. There is no precedent for how to establish a WebSocket connection from a HTTP/2 connection. That means, as far as I know, that no browser will connect to a wss:// URL and negotiate HTTP/2 within it: they will always negotiate HTTP/1.1. That should mean, again, that in practice having HTTP/2 enabled for Autobahn/Crossbar is just a no-op: it doesn't affect websockets at all.

glyph commented 8 years ago

I only care about latency until WebSocket takes over.

My point about latency is that you may be serving both traditional-web-application and websocket/autobahn interfaces for the same application. I understand that you could do this a different way, but keeping all your services the same can really simplify deployment pipelines, and it's definitely something that Twisted works hard to make possible, and Autobahn specifically allows for.

But, as @Lukasa explained - there's no overhead beyond the initial TLS handshake for websocket connections anyway, since wss:// clients won't advertise an ability to speak HTTP/2.

oberstet commented 8 years ago

Alright, here is an update:

I think supporting both directions for integration between Twisted and Autobahn like in above is a good approach and should cover most users needs in this area.

Of course, both features internally use "hacks" (unsupported, non-official "API" in Twisted) - and we should use official Twisted machinery once it is there. For now, "it works" ..

I close this now - if there are still open aspects, please come up, we can reopen should there be any.