Open howardjohn opened 2 months ago
cc @alyssawilk
Is this one of those "two passes through Envoy" issues?
The HTTP connection pool does properly handle goaways by marking the connection as draining and not creating new streams on that connection https://github.com/envoyproxy/envoy/blob/main/source/common/http/conn_pool_base.cc#L100
That said that HTTP/2 connection has no idea it's handling HTTP/1.1 as the inner payload so can't send a connection: close header. I think that'd have to be done by whoever is sending the HTTP/1.1. If you want to handle it in Envoy you could potentially advise open streams that a connection is draining and propagate that through the internal listener code but that's a feature I suspect you'd have to pick up yourselves.
I don't think I necessarily need connection:close onto the HTTP 1/1 connections.
So we have
app -- HTTP/1.1 ---> Envoy (Handling HTTP with an HCM) ---Internal Listener--> Same Envoy process ---> HTTP/2 CONNECT ---> Backend Proxy server (Terminates CONNECT) ---HTTP/1.1 --> backend app
When the "Backend Proxy server" is shutting down, it will send a GOAWAY over the outer HTTP/2 CONNECT, wait N seconds, then close the connection.
During this time between the GOAWAY and connection close ("draining"), I would want Envoy to not use the draining CONNECT connection for new HTTP/1.1 requests, and instead open up a new CONNECT request (or use an existing one, if there is another one already).
yeah so you'd want to tweak the codec client to warn all streams when the connection is draining, propogate that through the internal listener, and use that as a drain signal for the HTTP/1 "connection"
Title: Graceful shutdown with HTTP2 CONNECT
Description: We use Envoy to tunnel HTTP traffic over HTTP2 CONNECT.
When the server is shutting down, it will send us a GOAWAY. However, Envoy will still treat the connection as live.
What we want is for the inner-HTTP to stop using the HTTP2 CONNECT transport when it gets a GOAWAY, and only send new inner-http requests on a new CONNECT tunnel.