Cleartext HTTP/2 connections

[ioquatix]

I'm doing simple experiments with different web servers regarding HTTP2 and header casing (related to Rack 3/rackup), and thought I'd try falcon.

I can't seem to get falcon to speak HTTP2 via curl. I see the wiki linked up-thread is gone, and I didn't see anything terribly specific to this situation in the falcon project-page docs.

Should any of the following result in an HTTP2 response?

(Aside, can falcon talk HTTP1.1 and HTTP2 together, on the same endpoint?)

$ cat basic.ru
run { |env| [200, {'Bad-Header' => 'oops'}, ["Hello World"]] }

$ falcon --verbose serve --bind http://localhost:9292 -c basic.ru

  0.0s     info: Falcon::Command::Serve [oid=0x7e4] [ec=0x7f8] [pid=58008] [2024-06-08 20:01:02 -0700]
               | Falcon v0.47.6 taking flight! Using Async::Container::Forked {:count=>8, :restart=>true}.
               | - Binding to: #<Falcon::Endpoint http://localhost:9292/ {}>
               | - To terminate: Ctrl-C or kill 58008
               | - To reload configuration: kill -HUP 58008
 0.01s     info: Falcon::Service::Server [oid=0x834] [ec=0x7f8] [pid=58008] [2024-06-08 20:01:02 -0700]
               | Starting server on #<Falcon::Endpoint http://localhost:9292/ {}>

Run curl, asking for HTTP2.. curl sends an upgrade header, but the connection isn't upgraded and it receives an HTTP1.1 response.

 $ curl -v --http2 'http://localhost:9292'

* Host localhost:9292 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:9292...
* Connected to localhost (::1) port 9292
> GET / HTTP/1.1
> Host: localhost:9292
> User-Agent: curl/8.8.0
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQAoAAAAAIAAAAA
>
* Request completely sent off
< HTTP/1.1 200 OK
< bad-header: oops
< vary: accept-encoding
< content-length: 11
<
* Connection #0 to host localhost left intact
Hello World 

Since I'm unsure if/how upgrade works, I thought I'd forgo it and directly talk HTTP2, but falcon doesn't like that. I notice the up-thread comment there is no specification-compliant way to detect direct HTTP2, so maybe this shouldn't be expected to work? Curl kindly blames the server. :-) (On the Falcon-side, it's the same protocol error as up-thread.)

 $ curl -v --http2-prior-knowledge 'http://localhost:9292'
* Host localhost:9292 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:9292...
* Connected to localhost (::1) port 9292
* [HTTP/2] [1] OPENED stream for http://localhost:9292/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: http]
* [HTTP/2] [1] [:authority: localhost:9292]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.8.0]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: localhost:9292
> User-Agent: curl/8.8.0
> Accept: */*
>
* Request completely sent off
* Remote peer returned unexpected data while we expected SETTINGS frame.  Perhaps, peer does not support HTTP/2 properly.
* Recv failure: Connection reset by peer
* Failed receiving HTTP2 data: 56(Failure when receiving data from the peer)
* Closing connection
curl: (56) Remote peer returned unexpected data while we expected SETTINGS frame.  Perhaps, peer does not support HTTP/2 properly.

# Falcon

    5m     info: Async::HTTP::Protocol::HTTP1::Request: Reading HTTP/1.1 requests for Async::HTTP::Protocol::HTTP1::Server. [oid=0x910] [ec=0x924] [pid=58026] [2024-06-08 20:06:40 -0700]
               | Headers: {} from #<Addrinfo: [::1]:51205 TCP>
    5m     info: Async::HTTP::Protocol::HTTP1::Request: PRI * from #<Addrinfo: [::1]:51205 TCP> [oid=0x910] [ec=0x924] [pid=58026] [2024-06-08 20:06:40 -0700]
               | Responding with: 200 {"bad-header"=>["oops"], "vary"=>["accept-encoding"]}; #<Protocol::Rack::Body::Enumerable length=11 body=Array> | #<Async::HTTP::Body::Statistics sent 11 bytes; took 0.73ms in total; took 0.69ms until first chunk>
    5m     warn: Async::Task: PRI * from #<Addrinfo: [::1]:51205 TCP> [oid=0x938] [ec=0x924] [pid=58026] [2024-06-08 20:06:40 -0700]
               | Task may have ended with unhandled exception.
               |   Protocol::HTTP1::InvalidRequest: "SM"
               |   → vendor/bundle/ruby/3.3.0/gems/protocol-http1-0.19.1/lib/protocol/http1/connection.rb:190 in `read_request_line'
               |     vendor/bundle/ruby/3.3.0/gems/protocol-http1-0.19.1/lib/protocol/http1/connection.rb:197 in `read_request'
               |     vendor/bundle/ruby/3.3.0/gems/async-http-0.66.3/lib/async/http/protocol/http1/request.rb:14 in `read'
               |     vendor/bundle/ruby/3.3.0/gems/async-http-0.66.3/lib/async/http/protocol/http1/server.rb:29 in `next_request'
               |     vendor/bundle/ruby/3.3.0/gems/async-http-0.66.3/lib/async/http/protocol/http1/server.rb:50 in `each'
               |     vendor/bundle/ruby/3.3.0/gems/async-http-0.66.3/lib/async/http/server.rb:50 in `accept'
               |     vendor/bundle/ruby/3.3.0/gems/io-endpoint-0.10.3/lib/io/endpoint/wrapper.rb:178 in `block in accept'
               |     vendor/bundle/ruby/3.3.0/gems/async-2.12.0/lib/async/task.rb:164 in `block in run'
               |     vendor/bundle/ruby/3.3.0/gems/async-2.12.0/lib/async/task.rb:377 in `block in schedule'

Finally, since I don't know the details of Upgrade:, I wondered if multiple requests might convince curl and falcon to switch to HTTP2 (i.e. perhaps the server is permitted to reply with HTTP1.1 and then upgrade on subsequent requests, but since there are none, curl closes the connection and doesn't bother). This didn't work either, as in the first case, I received two HTTP/1.1 responses. I guess curl explains can not multiplex, even if we wanted to because the connection wasn't upgraded to HTTP2.

$ curl -v --http2 'http://localhost:9292' 'http://localhost:9292'
* Host localhost:9292 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:9292...
* Connected to localhost (::1) port 9292
> GET / HTTP/1.1
> Host: localhost:9292
> User-Agent: curl/8.8.0
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQAoAAAAAIAAAAA
>
* Request completely sent off
< HTTP/1.1 200 OK
< bad-header: oops
< vary: accept-encoding
< content-length: 11
<
* Connection #0 to host localhost left intact
Hello World* Found bundle for host: 0x600003130270 [serially]
* Can not multiplex, even if we wanted to
* Re-using existing connection with host localhost
> GET / HTTP/1.1
> Host: localhost:9292
> User-Agent: curl/8.8.0
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQAoAAAAAIAAAAA
>
* Request completely sent off
< HTTP/1.1 200 OK
< bad-header: oops
< vary: accept-encoding
< content-length: 11
<
* Connection #0 to host localhost left intact

Originally posted by @richardkmichael in https://github.com/socketry/falcon/issues/116#issuecomment-2156294549

[ioquatix]

Let me cross-reference https://github.com/socketry/async-http/pull/128 - we should try to get that merged.

[ioquatix]

I've merged clear-text HTTP/2 support, and I've added documentation about how to use it: https://github.com/socketry/async-http/tree/main/examples/hello

Do you mind testing it and reporting back?

[richardkmichael]

This works; I ran the new async-http example and tried v0.67.0 in my own experiments. Thank you!

[ioquatix]

Awesome, do you mind telling me a bit more about what you are doing?

[richardkmichael]

Sure, appreciate you asking! It's not terribly interesting. :)

I was using a minimal Rackup config to experiment with Rack::MiniProfiler, and bumped into Rack::Lint enforcing lowercase headers. I was unaware of header-related changes to the Rack and HTTP/2 specifications, so I was investigating.

I added a trivial middleware to lowercase headers, but Firefox Devtools still showed mixed-case headers (even raw). I was testing combinations of servers, HTTP versions and clients to determine which part of the stack modified the headers after my middleware.

Specifically regarding cleartext HTTP/2, I was using Wireshark to inspect the packets and misunderstood the io-endpoint SSL error (related to Falcon's self-signed cert). Being already down a few rabbit holes, I wanted to sidestep another. :-) (Debugging ruby/SSL.)

I also learned few unrelated bundler tidbits from your async-http example. Thank you very much!

[ioquatix]

That all sounds very interesting. Honestly, it's great to hear about your rabbit hole(s). Glad it was helpful and we could improve a few things :)