search

envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.19k stars 4.67k forks source link

Clarify behavior after max_connection_time timeout #34384

Open ktalg opened 3 weeks ago

ktalg commented 3 weeks ago

Title: One line description when max_connection_time and drain_timeout reached,the http2 connection wouldn't be closed if there's any active stream, which does not match what doc says

Description:

according to: https://www.envoyproxy.io/docs/envoy/v1.19.5/api-v3/config/core/v3/protocol.proto#config-core-v3-httpprotocoloptions

When max_connection_duration is reached the connection will be closed. Drain sequence will occur prior to closing the connection if if’s applicable.

But recently I ran a test and looked at the client trace logs:

2024-05-28T07:10:57.597366Z     debug   envoy http2     [C107] updating connection-level initial window size to 268435456
2024-05-28T07:10:57.597367Z     debug   envoy client    [C107] connecting
2024-05-28T07:10:57.597369Z     debug   envoy connection        [C107] connecting to 172.16.29.45:50051
...
2024-05-28T07:10:58.601368Z     trace   envoy http2     [C107] about to recv frame type=7, flags=0, stream_id=0
2024-05-28T07:10:58.601371Z     trace   envoy http2     [C107] track inbound frame type=7 flags=0 length=8 padding_length=0
2024-05-28T07:10:58.601374Z     trace   envoy http2     [C107] recv frame type=7
2024-05-28T07:10:58.601377Z     debug   envoy pool      [C107] remote goaway
2024-05-28T07:10:58.601383Z     trace   envoy http2     [C107] dispatched 17 bytes
2024-05-28T07:11:03.602011Z     trace   envoy connection        [C107] socket event: 3
2024-05-28T07:11:03.602027Z     trace   envoy connection        [C107] write ready
2024-05-28T07:11:03.602032Z     trace   envoy connection        [C107] read ready. dispatch_buffered_data=false
2024-05-28T07:11:03.602041Z     trace   envoy connection        [C107] read returns: 17
2024-05-28T07:11:03.602049Z     trace   envoy connection        [C107] read error: Resource temporarily unavailable
2024-05-28T07:11:03.602056Z     trace   envoy http2     [C107] dispatching 17 bytes
2024-05-28T07:11:03.602061Z     trace   envoy http2     [C107] about to recv frame type=7, flags=0, stream_id=0
2024-05-28T07:11:03.602063Z     trace   envoy http2     [C107] track inbound frame type=7 flags=0 length=8 padding_length=0
2024-05-28T07:11:03.602065Z     trace   envoy http2     [C107] recv frame type=7
...
2024-05-28T07:11:27.626089Z     trace   envoy http2     [C107] dispatched 27 bytes
2024-05-28T07:11:33.836496Z     debug   envoy client    [C107] request reset
2024-05-28T07:11:33.836504Z     debug   envoy pool      [C107] destroying stream: 0 remaining
2024-05-28T07:11:33.836511Z     debug   envoy connection        [C107] closing data_to_write=0 type=1
2024-05-28T07:11:33.836512Z     debug   envoy connection        [C107] closing socket: 1
2024-05-28T07:11:33.836544Z     trace   envoy connection        [C107] raising connection event 1
2024-05-28T07:11:33.836569Z     debug   envoy client    [C107] disconnect. resetting 0 pending requests
2024-05-28T07:11:33.836575Z     debug   envoy pool      [C107] client disconnected, failure reason:

After the second goaway, the server does not actively close the connection (stream exists) until the client closes (30 seconds later). Is this behavior correct? Or maybe I missed something.

zuercher commented 2 weeks ago

The docs you linke are quite old, v1.19.5. The current docs say, for a downstream connection, that if there are active streams the drain sequence is initiated. (https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/protocol.proto#config-core-v3-httpprotocoloptions)

ktalg commented 2 weeks ago

The docs you linke are quite old, v1.19.5. The current docs say, for a downstream connection, that if there are active streams the drain sequence is initiated. (https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/protocol.proto#config-core-v3-httpprotocoloptions)

Thank you. The latest version of the documentation state:

max_connection_duration:

If the connection is a downstream connection and there are any active streams, the drain sequence will kick in, and the connection will be force-closed after the drain period.

and

drain_timeout:

The time that Envoy will wait between sending an HTTP/2 “shutdown notification” (GOAWAY frame with max stream ID) and a final GOAWAY frame. Draining occurs both when a connection hits the idle timeout or during general server draining.

This still does not resolve my confusion. If a TCP connection has continuously active streams, after both max_connection_duration and drain_timeout have expired, will the stream be be force-closed, or will Envoy send two GOAWAY frames and then wait for certain conditions (hits the idle timeout or during general server draining) before closing the connection?

phlax commented 2 weeks ago

@ktalg if you were able to raise a PR to clarify the docs i would be happy to review