This PR is meant to introduce HTTP/2 streams and basic functionality of request / response handling.
What is working:
Protocol switching for both H2 and H2C methods. It is possible to initialize a new connection and handle incoming frames through handleHTTP2Connection which follows a similar approach to the corresponding HTTP/1 methods in http1.d. (handleHTTP2Connection -> handleHTTP2FrameChain -> handleHTTP2Frame)
The connection preface for the server is correctly initialized and sent, and the one sent by the client is correctly parsed and SETTINGS parameters are updated accordingly.
HEADERS, SETTINGS, DATA, PING frame handling is working, plus it is possible to respond to a basic HTTP/2 request (tested via curl --http2 https://localhost:8091). exchange.d contains methods used to parse a request and build a response, especially handleHTTP2request which is partially taken from originalHandleRequest, but adapted to parsing and writing HTTP/2 streams. The HTTP message exchange for HTTP/2 is documented in RFC 7540, Section 8 and it describes in depth how a HTTP/1.1 Start line (the message line containing the request method or the status for the response) can be converted to HTTP/2 pseudo-header representation. The function parseHTTP2RequestHeader in server.d and buildHeaderFrame in exchange.d implement this conversion.
Streams are represented by HTTP2ConnectionStream in http2.d, which embeds an underlying stream (be it TCPConnection for cleartext requests or a TLSStream for HTTPS. To implement its lifecycle (see RFC 7540, Section 5.1) an enum is used, but the complete implementation is currently WIP due to the missing blocks (below).
Some (minor) changes have been made to frame.d and the HPACK files due to the necessary adaptation to http.2. Furthermore, HTTP2Settings has been moved to settings.d along with the new HTTP2ServerContext structure.
Some minor changes have also been made to server.d and http1.d to allow for proper protocol switching. At the current state, http1.d is in charge of checking if an incoming request is HTTP/1 or HTTP/2. handleHTTP1Connection checks for incoming TLS streams which have been negotiated to use h2, while originalHandleRequest checks for Upgrade header fields and initializes protocol switching. The existing code (introduced in milestone 1) had to be modified in order to store the response to the first HTTP/1 request which contained the Upgrade header field, so that a HTTP/2 Stream with ID 1 could reply to it (see RFC 7540, Section 3)
The two unittest blocks in http.2 can be used to test functionality by uncommenting the runApplication call. The missing blocks are the following:
Proper stream multiplexing. Stream multiplexing currently works by using HTTP2ServerContext to manage streams IDs as they get assigned. This means that handling a single stream is perfectly feasible, and even completing a single request, but for more complex uses (multiple streams, dependent streams) the logic has yet to be introduced. The reason why this PR has been opened is to address that, and discuss of the current state (so that the logic can be introduced and tested without much hassle). Because of this, PRIORITY, RST_STREAM and GOAWAY frame handling is not yet implemented.
CONTINUATION frame handling is done, but not enabled ATM. The strategy that is used here is the following: whenever a HEADER frame is received without the END_HEADERS flag set, handleFrameAlloc gets called recursively to read the further CONTINUATION frame, until no more are found (END_HEADERS is set). This works but since multiplexing is not yet operational it might cause conflicts with the continuous polling performed by handleHTTP2FrameChain.
Flow control. This is done by using WINDOW_UPDATE frames, which set the maximum number of octets the endpoind (client in this case) is willing to receive. This should be fairly easy to introduce once multiplexing is up and running.
PUSH_PROMISE frames are postponed as discussed, but support for them will be added.
I know this PR is quite big,if this becomes a problem for the review I'm ok with splitting the changes in multiple PRs. Unfortunately I had to work on it until I had something logically coherent to submit and minimally functional so that obvious issues could be spotted easily. As a result, the proposed code can be tested with a client, it is able to send HTTP/2 Frames which are properly formed and to decode incoming requests thanks to HPACK.
Starting from here, the missing parts of the implementation could be built without too much hassle and properly tested. I would also like to concentrate on resource handling and note that unfortunately I wasn't able to respect @nogc in most of the code, mainly due to the lack of @nogc compatibility in vibe-http at the moment. I mean to work on that as soon as the HTTP/2 module is fully functional.
A note: testing became a problem especially due to the lack of complete testing suites apart from http2-test, which unfortunately doesn't seem to work properly with a server (its focus seems to be on client testing). At the current state a simple browser is enough to stress the implementation due to the complex management of PRIORITY Frames and stream dependencies, but simple clients i.e. curl or nghttp2 are supported.
Edit: I forgot to add it to the above description but I would like to discuss the best way to mantain the IndexingTable state during a whole connection. Right now it is initialized during each stream initialization, which might result in wrong decoding if multiple requests on multiple streams are sent on the same connection.
This PR is meant to introduce HTTP/2 streams and basic functionality of request / response handling.
What is working:
handleHTTP2Connection
which follows a similar approach to the corresponding HTTP/1 methods inhttp1.d
. (handleHTTP2Connection
->handleHTTP2FrameChain
->handleHTTP2Frame
) The connection preface for the server is correctly initialized and sent, and the one sent by the client is correctly parsed and SETTINGS parameters are updated accordingly.curl --http2 https://localhost:8091
).exchange.d
contains methods used to parse a request and build a response, especiallyhandleHTTP2request
which is partially taken fromoriginalHandleRequest
, but adapted to parsing and writing HTTP/2 streams. The HTTP message exchange for HTTP/2 is documented in RFC 7540, Section 8 and it describes in depth how a HTTP/1.1 Start line (the message line containing the request method or the status for the response) can be converted to HTTP/2 pseudo-header representation. The functionparseHTTP2RequestHeader
inserver.d
andbuildHeaderFrame
inexchange.d
implement this conversion.HTTP2ConnectionStream
inhttp2.d
, which embeds an underlying stream (be itTCPConnection
for cleartext requests or aTLSStream
for HTTPS. To implement its lifecycle (see RFC 7540, Section 5.1) anenum
is used, but the complete implementation is currently WIP due to the missing blocks (below).frame.d
and the HPACK files due to the necessary adaptation tohttp.2
. Furthermore,HTTP2Settings
has been moved tosettings.d
along with the newHTTP2ServerContext
structure.server.d
andhttp1.d
to allow for proper protocol switching. At the current state,http1.d
is in charge of checking if an incoming request is HTTP/1 or HTTP/2.handleHTTP1Connection
checks for incoming TLS streams which have been negotiated to useh2
, whileoriginalHandleRequest
checks forUpgrade
header fields and initializes protocol switching. The existing code (introduced in milestone 1) had to be modified in order to store the response to the first HTTP/1 request which contained theUpgrade
header field, so that a HTTP/2 Stream with ID 1 could reply to it (see RFC 7540, Section 3)The two
unittest
blocks inhttp.2
can be used to test functionality by uncommenting therunApplication
call. The missing blocks are the following:HTTP2ServerContext
to manage streams IDs as they get assigned. This means that handling a single stream is perfectly feasible, and even completing a single request, but for more complex uses (multiple streams, dependent streams) the logic has yet to be introduced. The reason why this PR has been opened is to address that, and discuss of the current state (so that the logic can be introduced and tested without much hassle). Because of this, PRIORITY, RST_STREAM and GOAWAY frame handling is not yet implemented.handleFrameAlloc
gets called recursively to read the further CONTINUATION frame, until no more are found (END_HEADERS is set). This works but since multiplexing is not yet operational it might cause conflicts with the continuous polling performed byhandleHTTP2FrameChain
.I know this PR is quite big,if this becomes a problem for the review I'm ok with splitting the changes in multiple PRs. Unfortunately I had to work on it until I had something logically coherent to submit and minimally functional so that obvious issues could be spotted easily. As a result, the proposed code can be tested with a client, it is able to send HTTP/2 Frames which are properly formed and to decode incoming requests thanks to HPACK.
Starting from here, the missing parts of the implementation could be built without too much hassle and properly tested. I would also like to concentrate on resource handling and note that unfortunately I wasn't able to respect
@nogc
in most of the code, mainly due to the lack of@nogc
compatibility invibe-http
at the moment. I mean to work on that as soon as the HTTP/2 module is fully functional.A note: testing became a problem especially due to the lack of complete testing suites apart from
http2-test
, which unfortunately doesn't seem to work properly with a server (its focus seems to be on client testing). At the current state a simple browser is enough to stress the implementation due to the complex management of PRIORITY Frames and stream dependencies, but simple clients i.e.curl
ornghttp2
are supported.Edit: I forgot to add it to the above description but I would like to discuss the best way to mantain the
IndexingTable
state during a whole connection. Right now it is initialized during each stream initialization, which might result in wrong decoding if multiple requests on multiple streams are sent on the same connection.