StackExchange / StackExchange.Redis

General purpose redis client
https://stackexchange.github.io/StackExchange.Redis/
Other
5.88k stars 1.51k forks source link

Connections: detect and handle the Linux dead socket case #2610

Closed NickCraver closed 10 months ago

NickCraver commented 10 months ago

In Linux, we see 15 minute socket stalls due to OS-level TCP retries. What this means is the PhysicalConnection detects no issues on the pipe that's retrying, but is also not receiving data at all leading to long stalls in client applications. The goal here is to detect that we're timing out commands people have issued to the connection but we're getting NOTHING back on the socket at all. In this case, we should assume the socket is dead and issue a reconnect so that we get out of the hung situation much faster.

For an initial go at this, we've chosen 4x the timeout interval as a threshold, but could make this configurable if needed.