rpxy accepts HTTP2 when force_http11_upstream is set, request fails

[akostadinov]

Don't know if this is related to #77 or not. But I have an upstream that only supports HTTP 1.1

Like this:

[apps.myapp]
server_name = 'myrpxy.example.com'
reverse_proxy = [{ upstream = [{ location = 'OpenWrt', tls = true }] }]
tls = { https_redirection = true, acme = true }
upstream_options = [ "force_http11_upstream" ]

But when running curl:

curl -v https://myrpxy.example.com
* Host myrpxy.example.com:443 was resolved.
* IPv6: (none)
* IPv4: 192.168.1.124
*   Trying 192.168.1.124:443...
* Connected to myrpxy.example.com (192.168.1.124) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=myrpxy.example.com
*  start date: Sep  2 20:47:06 2024 GMT
*  expire date: Dec  1 20:47:05 2024 GMT
*  subjectAltName: host "myrpxy.example.com" matched cert's "myrpxy.example.com"
*  issuer: C=US; O=Let's Encrypt; CN=E6
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://myrpxy.example.com/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: myrpxy.example.com]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.6.0]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: myrpxy.example.com
> User-Agent: curl/8.6.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/2 500 
< date: Sun, 15 Sep 2024 22:39:18 GMT
< 
* Connection #0 to host myrpxy.example.com left intact

So basically TLS handshake appears to incorrectly accept h2 when it should accept http/1.1 because of the upstream server.

P.S. Request works with curl --http1.1 but there is no such option for normal browsers.

[junkurihara]

force_http11_upstream option just converts incoming requests of HTTP/1.1, 2 or 3 to that of HTTP/1.1, and does not offers only HTTP/1.1 directly to clients.

In my understanding, your setting seems as below.

client -> (https over http/2) -> rpxy -> (https over http/1.1) -> backend app

But it is weird that your application returns 500 when clients send request with HTTP/2. The setting seems that rpxy converts HTTP/2 to HTTP/1.1 towards your backend app. So could you check logs of rpxy and your backend app?

[akostadinov]

Sorry, I missed to paste rpxy log because the error message in rpxy log seemed like http2 forwarding to http1 was not possible. I think the relevant part is:

2024-09-16T20:38:45.038091Z DEBUG rpxy rpxy_lib::proxy::proxy_main:191: HTTP/2 or 1.1: SNI in ClientHello: "myserver.example.com"
2024-09-16T20:38:45.060956Z DEBUG rpxy rpxy_lib::proxy::proxy_main:82: Request incoming: current # 2
2024-09-16T20:38:45.082350Z DEBUG rpxy rpxy_lib::backend::upstream:88: Found upstream: "/"
2024-09-16T20:38:45.083015Z DEBUG rpxy rpxy_lib::message_handler::handler_manipulate_messages:69: Generate request to be forwarded
2024-09-16T20:38:45.083664Z DEBUG rpxy rpxy_lib::backend::upstream:231: Upstream of index 0 is chosen.
2024-09-16T20:38:45.083690Z DEBUG rpxy rpxy_lib::backend::upstream:232: Context to LB (Cookie in Request): None
2024-09-16T20:38:45.083697Z DEBUG rpxy rpxy_lib::backend::upstream:233: Context from LB (Set-Cookie in Response): None
2024-09-16T20:38:45.083710Z DEBUG rpxy rpxy_lib::message_handler::handler_main:163: Request to be forwarded: [uri https://OpenWrt/, method: GET, version HTTP/2.0, headers {"user-agent": "curl/8.6.0", "accept": "*/*", "x-forwarded-for": "192.168.1.203", "x-forwarded-proto": "https", "x-forwarded-port": "8443", "x-real-ip": "192.168.1.203", "x-forwarded-ssl": "on", "x-original-uri": "https://myserver.example.com/", "proxy": "", "host": "myserver.example.com"}]
2024-09-16T20:38:45.308347Z  WARN rpxy hyper_util::client::legacy::client:283: Connection is HTTP/1, but request requires HTTP/2
2024-09-16T20:38:45.310803Z ERROR rpxy rpxy_lib::message_handler::handler_main:74: Failed to get response from backend: Failed to fetch from upstream: client error (UserUnsupportedVersion)
2024-09-16T20:38:45.313777Z  INFO rpxy rpxy_lib::message_handler::http_log:76: myserver.example.com <- 192.168.1.203:56784 -- GET / HTTP/2.0 -- 500 Internal Server Error -- https://myserver.example.com "curl/8.6.0", "192.168.1.203" "https://OpenWrt/"
2024-09-16T20:38:45.350427Z DEBUG rpxy rpxy_lib::proxy::proxy_main:112: Request processed: current # 1

More specifically

Connection is HTTP/1, but request requires HTTP/2

P.S. the upstream service is OpenWRT whatever its configuration interface uses as a web server. But should be pretty simple software because these devices are resources constraint. I expect something that adheres to as few modern specifications as possible.