Support HTTP/2 (httpie-http2 plugin doesn't work)

[maximbaz]

It's 2018 now, HTTP/2 is widely used, but httpie still doesn't support it out of the box. Let's fix this.

I know about httpie/httpie-http2, but there are issues with this approach:

1. httpie/httpie-http2 doesn't come out of the box when httpie gets installed, many people would just never know that this plugin even exists.
2. Linux distributions do not provide a package for that plugin.
3. People have issues installing the plugin on macOS: https://github.com/jakubroztocil/httpie/issues/298#issuecomment-273360856.
4. The entire code of httpie/httpie-http2 is just 20 lines (7 useful lines), integration will be trivial.
5. Even curl supports HTTP/2 out of the box.

I see many benefits of supporting HTTP/2 natively. What are the downsides?

[maximbaz]

I just tried to go through installation of the httpie/http2 plugin and it appears to be impossible to use it right now. The plugin depends on hyper project, which in turn depends on a very old version of hyperframe project. This old hyperframe is not available for my Arch Linux, so I cannot install hyper and thus httpie/http2 (well, I can install, but it crashes in runtime).

The request to update hyper and its dependencies is open since a year ago, with no response from the hyper's author. It seems to be abandoned.

Perhaps this issue should be rephrased to "implement support for HTTP/2". At this moment it is simply impossible to use HTTP/2 in any way via httpie.

[inductor]

@jakubroztocil Any thoughts? http2 plugin has lots of problems and it seems like nobody is maintaining that repo

[Frodox]

bump @jakubroztocil

[northrup]

@jakubroztocil hitting this up again as I had a need to test http/2 from a CLI but had to fall back to curl to get it done :/

[Bad3r]

same issue i guess curl will always be better its been more than year.

[jkbrzt]

I’d also love to have built-in HTTP2 support.

The httpie/httpie-http2 plugin is built on top python-hyper/hyper. That is the only HTTP2 solution for psf/requests I’m aware of and, sadly, it doesn’t seem to be actively maintained any longer.

I’m keeping an eye on encode/httpx. It comes with a built-in HTTP2 support and might become a replacement for requests at some point.

In any case, I’ll start by trying to fix the plugin. If successful, then I’ll look into bundling it with HTTPie. The general plugin installation issues will be addressed as well.

(Cc. @Lukasa @tomchristie @florimondmanca)

[Lukasa]

Yeah, hyper is basically unmaintained: it got superseded by hyper-h2 and the associated other building blocks, and simply stopped being the most important thing. Httpx is definitely the thing to keep an eye on.

[tomchristie]

I'd like to pick this up again - I'm planning on releasing httpx/httpcore 1.0 this month, and after that getting HTTP/2 support for httpie is right up there on my list. Because httpie has a super large install base it's going to be really important to minimise any risk of unintended changes in the network behaviour, even for edge cases etc.

I think the approach in #972 is the smart route to take - continue using requests, but switch the underlying adapter to httpcore in order to support HTTP/2. I'll have another look at reissuing a pull request for that once httpcore 1.0 is released.

I would probably suggest that the cautious approach would (at least to start with) be to only enable HTTP/2 support when explicitly requested(?) Ie. have an --http2 flag, and only switch to httpcore in that case. The benefit of this is that all existing behaviour will stay the same, which just means a much lower risk factor for any changes.

One other fiddly bits:

• --http2 should imply --default-scheme=https. Or perhaps error unless it's used with the https command? (Because HTTP/2 is normally HTTPS-only)
• When HTTP/2 support is enabled you don't actually know if the request will end up being HTTP/2 or HTTP/1.1 until the SSL connection has been established. This means you'll sometimes end up with the following kind of behaviour...

> https www.http11onlyserver.com --http2
GET / HTTP/2  # <--- Prints as if an HTTP/2 request was sent.
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Host: www.http11onlyserver.com
User-Agent: HTTPie/2.6.0

HTTP/1.1 200 OK  # <--- But actually the connection ended up on HTTP/1.1
Age: 500630
Cache-Control: max-age=604800
Content-Encoding: gzip
Content-Length: 648
Content-Type: text/html; charset=UTF-8
Date: Thu, 16 Dec 2021 10:32:28 GMT
Etag: "3147526947+gzip"
Expires: Thu, 23 Dec 2021 10:32:28 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (nyb/1D04)
Vary: Accept-Encoding
X-Cache: HIT

That can only be resolved by using some hooks to httpcore that provide detailed callbacks, and printing the request once the connection is actually established. But that'd require a much bigger change to httpie, and doesn't seem worth the complexity for a first-pass.

(Although if you did ever go down that route there's some useful information you'd be able to start exposing from httpie, but that's a whole different conversation.)

Anyways - any thoughts? Does this seem like a reasonable approach?

[jcayzac]

It's 2018 now, HTTP/2 is widely used

It's 2022 now, HTTP/3 is an industry standard…

[tx0c]

the httpie/httpie-http2 plugin is just broken, does not work at all for anything, have to uninstall:

$ httpie --verbose https://h2book.com
GET / HTTP/1.1
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
User-Agent: HTTPie/3.1.0
Host: h2book.com

__main__.py: error: TypeError: sequence item 0: expected str instance, bytes found

[earwin]

@tomchristie, just as a heads-up: there are servers which only support HTTP/2, so 1.1 -> 2 upgrade fails. curl has an (awkwardly named) --http2-prior-knowledge key to accomodate for this

[tomchristie]

@earwin - Sure. I'm not doing any work on HTTP/2 in httpie at the moment. If there's interest from the team, then I'm potentially up for spending time on it.

For now you can use curl or the httpx command-line client for HTTP/2 requests.