vibe-d / vibe-http

Future vibe.d HTTP implementation
15 stars 9 forks source link

Stream functionality and HTTP/2 request / response exchange #9

Closed GallaFrancesco closed 5 years ago

GallaFrancesco commented 6 years ago

This PR is meant to introduce HTTP/2 streams and basic functionality of request / response handling.

What is working:

  1. Protocol switching for both H2 and H2C methods. It is possible to initialize a new connection and handle incoming frames through handleHTTP2Connection which follows a similar approach to the corresponding HTTP/1 methods in http1.d. (handleHTTP2Connection -> handleHTTP2FrameChain -> handleHTTP2Frame) The connection preface for the server is correctly initialized and sent, and the one sent by the client is correctly parsed and SETTINGS parameters are updated accordingly.
  2. HEADERS, SETTINGS, DATA, PING frame handling is working, plus it is possible to respond to a basic HTTP/2 request (tested via curl --http2 https://localhost:8091). exchange.d contains methods used to parse a request and build a response, especially handleHTTP2request which is partially taken from originalHandleRequest, but adapted to parsing and writing HTTP/2 streams. The HTTP message exchange for HTTP/2 is documented in RFC 7540, Section 8 and it describes in depth how a HTTP/1.1 Start line (the message line containing the request method or the status for the response) can be converted to HTTP/2 pseudo-header representation. The function parseHTTP2RequestHeader in server.d and buildHeaderFrame in exchange.d implement this conversion.
  3. Streams are represented by HTTP2ConnectionStream in http2.d, which embeds an underlying stream (be it TCPConnection for cleartext requests or a TLSStream for HTTPS. To implement its lifecycle (see RFC 7540, Section 5.1) an enum is used, but the complete implementation is currently WIP due to the missing blocks (below).
  4. Some (minor) changes have been made to frame.d and the HPACK files due to the necessary adaptation to http.2. Furthermore, HTTP2Settings has been moved to settings.d along with the new HTTP2ServerContext structure.
  5. Some minor changes have also been made to server.d and http1.d to allow for proper protocol switching. At the current state, http1.d is in charge of checking if an incoming request is HTTP/1 or HTTP/2. handleHTTP1Connection checks for incoming TLS streams which have been negotiated to use h2, while originalHandleRequest checks for Upgrade header fields and initializes protocol switching. The existing code (introduced in milestone 1) had to be modified in order to store the response to the first HTTP/1 request which contained the Upgrade header field, so that a HTTP/2 Stream with ID 1 could reply to it (see RFC 7540, Section 3)

The two unittest blocks in http.2 can be used to test functionality by uncommenting the runApplication call. The missing blocks are the following:

I know this PR is quite big,if this becomes a problem for the review I'm ok with splitting the changes in multiple PRs. Unfortunately I had to work on it until I had something logically coherent to submit and minimally functional so that obvious issues could be spotted easily. As a result, the proposed code can be tested with a client, it is able to send HTTP/2 Frames which are properly formed and to decode incoming requests thanks to HPACK.

Starting from here, the missing parts of the implementation could be built without too much hassle and properly tested. I would also like to concentrate on resource handling and note that unfortunately I wasn't able to respect @nogc in most of the code, mainly due to the lack of @nogc compatibility in vibe-http at the moment. I mean to work on that as soon as the HTTP/2 module is fully functional.

A note: testing became a problem especially due to the lack of complete testing suites apart from http2-test, which unfortunately doesn't seem to work properly with a server (its focus seems to be on client testing). At the current state a simple browser is enough to stress the implementation due to the complex management of PRIORITY Frames and stream dependencies, but simple clients i.e. curl or nghttp2 are supported.

Edit: I forgot to add it to the above description but I would like to discuss the best way to mantain the IndexingTable state during a whole connection. Right now it is initialized during each stream initialization, which might result in wrong decoding if multiple requests on multiple streams are sent on the same connection.

GallaFrancesco commented 5 years ago

Closing this to rebase and split. The changes will be proposed incrementally in a few PRs coming.