search

hyperium / h2

HTTP 2.0 client & server implementation for Rust.
MIT License
1.35k stars 270 forks source link

"protocol error: unspecific protocol error detected" making http2 request to go.googlesource.com #548

Closed kevinburke closed 2 years ago

kevinburke commented 3 years ago

I compiled curl with rustls-ffi and Hyper. I'm attempting to make a request to https://go.googlesource.com. Here's what I get when I try that:

$ curl --version
curl 7.78.0 (x86_64-apple-darwin19.6.0) libcurl/7.78.0 crustls/0.7.1/rustls/0.19.0 zlib/1.2.11 brotli/1.0.9 zstd/1.5.0 libidn2/2.3.2 librtmp/2.3 Hyper/0.14.11 OpenLDAP/2.5.5
Release-Date: 2021-07-21
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli Debug GSS-API HSTS HTTP2 IDN IPv6 Kerberos Largefile libz SPNEGO SSL TrackMemory UnixSockets zstd

$ curl https://go.googlesource.com
curl: (56) Hyper: [1] error reading a body from connection: protocol error: unspecific protocol error detected

Full -vvv logs are available here: https://gist.github.com/kevinburke/c2ac5cf3ed7f59c650263288010ccdce

I'm compiling curl on Mac using the Homebrew formula available at github.com/meterup/homebrew-safe. You can install using brew install meterup/safe/curl.

I tracked down that "unspecific protocol error" message to this library, so I'm hopeful that this is the right place to file a ticket. I'm also hopeful that you are able to reproduce the issue.

I'm not sure what your policy is on h2 handling HTTP/2 implementations in the wild... it's possible that the server is doing something wrong, but I don't know enough about HTTP/2 to say for certain. The server is running Gitiles (https://gerrit.googlesource.com/gitiles/), though I'm not sure if that service is also terminating TLS, or if there's some other layer in between that's handling that.

Updates rustls/rustls-ffi#129.

seanmonstar commented 3 years ago

I'm all for improving interop with HTTP2 implementations. So, first thing I did was try this using a pure Rust implementation. I used reqwest's simple example, with default-tls disabled and rustls enabled, so it would use ALPN and HTTP2. It works correctly for me... I get a 200 OK response, and the full HTML body.

Does this happen with other hosts that use HTTP2?

kevinburke commented 3 years ago

I can successfully make HTTP2 requests with my rustls + h2 + curl to fastly.com, meterapi.com (Amazon ALB), www.amazon.com, cloudflare.com. I am happy to test others if you know other independent implementations.

I can't appear to make connections to GCP servers (golang.org, console.cloud.google.com, go.googlesource.com)

seanmonstar commented 3 years ago

Do you know what the version is of the h2 dependency? When building, there should be a Cargo.lock file generated, and that should contain the version info.

kevinburke commented 3 years ago

I'm using Hyper v0.14.11 - you can see the Homebrew formula here: https://github.com/meterup/homebrew-safe/blob/main/Formula/hyper.rb

This is the Cargo.lock for Hyper v0.14.11:

[[package]]
name = "h2"
version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "825343c4eef0b63f541f8903f395dc5beb362a979b5799a84062527ef1e37726"
dependencies = [
 "bytes",
 "fnv",
 "futures-core",
 "futures-sink",
 "futures-util",
 "http",
 "indexmap",
 "slab",
 "tokio",
 "tokio-util",
 "tracing",
]
seanmonstar commented 3 years ago

Huh ok, so the latest one. Does curl have an ability to print what bytes are sent and received on the socket? Or maybe something like Wireshark would help.

divergentdave commented 3 years ago

I encountered similar issues with curl/rustls/hyper on https://www.google.com/. From my observations, I think the Google Front End servers are picky about headers/pseudo-headers on HTTP/2 connections, and they will return this error if either they (a) see a Host header or (b) don't see :scheme: https and :authority set appropriately.

I have some WIP branches to fix these issues, see hyperium/hyper#2581 and https://github.com/divergentdave/curl/tree/hyper-http2-scheme-and-authority. (Looks like I need to rebase the hyper PR, and last I worked on it, my curl branch had mysterious Valgrind-only issues) Building against these two, ./src/curl https://www.google.com/ returns "unspecific protocol error detected", and ./src/curl -H 'Host:' https://www.google.com/ works, and https://go.googlesource.com/ behaves the same way.

Long story short, it's primarily a curl bug, and GFE is the strictest server out there about headers over HTTP/2 connections.

kevinburke commented 2 years ago

This can be closed - https://github.com/curl/curl/issues/7679 and hyperium/hyper#2581 fix the issue in question.