StackExchange / StackExchange.Redis

General purpose redis client
https://stackexchange.github.io/StackExchange.Redis/
Other
5.88k stars 1.51k forks source link

Keep-alive fails periodically while other clients operate fine #2613

Closed asafchen-dig closed 9 months ago

asafchen-dig commented 9 months ago

I'm having this error with the same values every few seconds. PING command from another clients work fine. Restarting the process (docker container) fixed it.

Do you know what was going on? and how can I keep it from happening again?

2023-12-07 09:15:34.1089 [ERROR] StackExchange.Redis.Extensions.Core.Implementations.RedisConnectionPoolManager - 
Redis internal error KeepAlive 
StackExchange.Redis.RedisTimeoutException: The timeout was reached before the message could be written to the 
output buffer, and it was not sent, 
command=PING, 
timeout: 5000, 
outbound: 0KiB, 
inbound: 0KiB, 
inst: 0, 
qu: 10, 
qs: 0, 
aw: True, 
bw: CheckingForTimeoutComplete, 
rs: ReadAsync, 
ws: Idle, 
in: 0, 
in-pipe: 0, 
out-pipe: 0,
last-in: 28, 
cur-in: 0, 
sync-ops: 0, 
async-ops: 2549, 
serverEndpoint: xxxxxxxxx:6379, 
conn-sec: 62220.9, 
aoc: 1, 
mc: 1/1/0, 
mgr: 2 of 2 available, 
clientName: xxxxxxxxxxxxx(SE.Redis-v2.7.4.20928), 
IOCP: (Busy=0,Free=1000,Min=1,Max=1000), 
WORKER: (Busy=3,Free=32764,Min=1,Max=32767), 
POOL: (Threads=8,QueuedItems=0,CompletedItems=1712305,Timers=27), 
v: 2.7.4.20928 
(Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)
NickCraver commented 9 months ago

This looks like a custom setup via StackExchange.Redis.Extensions.Core.Implementations.RedisConnectionPoolManager (which isn't this library/repo) that's also customizing things like the socket manager thread pool going by the 2 of 2 available in the exception. I'd recommend logging an issue against that library for this since it's doing some not-so-compatible things with socket managers and shared pools going by the behavior and names.

asafchen-dig commented 9 months ago

Thanks!