HTTP/2 support

[The-EDev]

in response to https://github.com/ipkn/crow/issues/47, https://github.com/ipkn/crow/issues/219, https://github.com/ipkn/crow/issues/292.

I'm not sure how to go about this, a good place to start implementing would be the comments of the issues themselves, and maybe we could also take a look at this to further understand how to implement the standard fully.

[The-EDev]

This is starting to seem like a rabbit hole with no end.

Ignoring the 49% adoption of HTTP/2 after 5 years of its release (not even announcement), and the controversy surrounding it and SPDY (with things like server push, and slowing load-balancers due to HPACK).

The http_parser we currently have does not support HTTP/2, and neither does the NGINX parser we proposed switching to, instead NGINX have a separate module to handle everything HTTP/2, which could be useful. But the point that it's a large pain, with lots of code to rewrite, for a seemingly slightly faster protocol that still relies on HTTP/1.1 to establish a connection.

Don't get me wrong, I do think HTTP/2 has a place, things like server push (while making me think about what servers can send without clients asking) have their place, and I would be working on HTTP/2 support 100% If we had 5 or 6 part time people supporting all the other features and bugs. But as it stands, I'm just not sure HTTP/2 is so important it's worth spending one or more months getting initial support working and then even more time maintaining 2 protocols instead of 1.

This is an email to the IETF that partially shaped my answer, the rest of the answer was shaped by the spec itself (mainly how much needs to be rewritten or changed to accommodate this new protocol that isn't very new or all that much faster than 1.1).

[NobodyXu]

@The-EDev I personally don't think HTTP/2 is important to crow due to reasons I mentioned in #88 .

I would say it is much simpler and more benefitial to implement fastCGI instead

[NobodyXu]

Besides, IETF is soon going to publish their HTTP/3 (QUIC) which many might want to migrate to as it improve performance of web under 4G and allows google to have more control over the stack since QUIC is based on UDP.

[NobodyXu]

http/1.1 is enough for simple application writen in crow and for programmers to debug their own code writen in crow.

[NobodyXu]

If user really wants to integrate their reverse proxy server and content server, we can make crow qualified to become a nginx plugin.

In this way, we 'get' http/2.0 (and possibly http/3.0) support for free.

[The-EDev]

Some new information I found out that makes HTTP/2 easier to set up (and even less useful):

HTTP/2 does not actually modify the structure of an HTTP message in any meaningful way, it only modifies the way a TCP connection is handled (multiplexing + server pushing) and adds a layer on top for sending data (using frames instead of just sending data directly through the TCP connection).

This means that the same HTTP parser can be used. but the res_push would need to be finalized and the http_connection object would need to be modified to accommodate the way HTTP/2 works

[patlecat]

Any updates on this or even HTTP/3 development?

[The-EDev]

unfortunately not, several other features take precedence over HTTP/2, things like asynchronous request handlers

[patlecat]

@The-EDev It's just that in 2023 I can't justify to use a non-HTTPS/2 or /3 solution anymore.

[The-EDev]

Well one way to get HTTP/2 support would be to connect Crow to Apache or Nginx as a reverse proxy, although since Crow's logic doesn't allow for things like duplexing the benefits would still be somewhat limited.

[patlecat]

@The-EDev Yes I have an idea how to get HTTP/2 without Crow ;) but seriously adopting HTTP/3 would be a smart move now methinks. It's coming.

[The-EDev]

I'll gladly help out with all my knowledge about Crow if anyone's willing to take on the work required to adopt HTTP/2 or 3, either through a separate library or as an addition to Crow, but with my current time it's very difficult to justify such a massive change that the majority of people don't actually notice.

[NobodyXu]

@patlecat You might as well use a different framework, such as axum or other frameworks that uses hyper, a http library that supports https and working on supporting http3.

[patlecat]

@NobodyXu Yeah but why would I want to use Rust? There aren't many easy to use HTTP-libs around for C++ sadly.

[NobodyXu]

@NobodyXu Yeah but why would I want to use Rust?

Because:

There aren't many easy to use HTTP-libs around for C++ sadly.

Plus writing Rust is a much more pleasant experience than C++, you won't have SIGSEGV unless you use unsafe.

Sum-type is really useful for expressing errors and different situations.

[patlecat]

I'll gladly help out with all my knowledge about Crow if anyone's willing to take on the work required to adopt HTTP/2 or 3, either through a separate library or as an addition to Crow, but with my current time it's very difficult to justify such a massive change that the majority of people don't actually notice.

Adapting to HTTP/2 or /3 isn't a user visible feature. Most people will never even understand what the differences are. But for companies serving thousands of requests per seconds this is an existiential difference.

[The-EDev]

@patlecat could you please elaborate on how HTTP/2 and HTTP/3 benefit larger scale applications?

[patlecat]

@The-EDev By the performance increase of bundling several requests and by the decreased time it needs to establish the initial handshakes. With a few dozen requests per second that may not make a big differences but with many thousands or more it adds up considerably and this would save time and money (not the least with HW). It's the reason why some companies install a proxy server only to process the HTTPS tasks that took too much time on the main app server. Roughly.

[ffreality]

@patlecat could you please elaborate on how HTTP/2 and HTTP/3 benefit larger scale applications?

As I know GraphQL's subscription feature require server push (tell me if I wrong) and we can implement our own subscription features.

[gfeyer]

Could we use nghttp2 for HTTP2 instead of building our own implementation? I'm thinking it might be easier to integrate an existing robust library instead of developing it from scratch.

[ffreality]

Could we use nghttp2 for HTTP2 instead of building our own implementation? I'm thinking it might be easier to integrate an existing robust library instead of developing it from scratch.

nghttp2 yes but I don't suggest libwebsocket because lack of custom header support. They integrated it only for http/1.1 and you have to know header's name.