StackExchange / StackExchange.Redis

General purpose redis client
https://stackexchange.github.io/StackExchange.Redis/
Other
5.88k stars 1.51k forks source link

Random StackExchange.Redis.RedisTimeoutException #2137

Closed nguyenluthuy249 closed 2 years ago

nguyenluthuy249 commented 2 years ago

Hi,

The following exception randomly occurs

Timeout awaiting response (outbound=0KiB, inbound=0KiB, 5828ms elapsed, timeout is 5000ms), inst: 0, qu: 0, qs: 1, aw: False, rs: ReadAsync, ws: Idle, in: 0, in-pipe: 0, out-pipe: 0, serverEndpoint: 10.10.3.24:6379, mgr: 10 of 10 available, clientName: APPSLC02, IOCP: (Busy=0,Free=1000,Min=6,Max=1000), WORKER: (Busy=1,Free=32766,Min=6,Max=32767), v: 2.0.593.37019 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)

Looking into the stats, I do not see any issue. The current setup is we have our web hosted in a Network Load Balancing including 2 webheads

  1. WEBAPPSLC02 - Redis is running on this server as well
  2. APPSLC02

The issue happened when a request from APPSLC02 reached to WEBAPPSLC02 getting something from cache. However, it randomly happened (1-2 times per day and usually we had hundreds of calls from APPSLC02 to WEBAPPSLC02 getting something from cache).

NickCraver commented 2 years ago

I'm not sure your scenario here, but you'll definitely want to be on the latest version which has drastically more information available to help us advise as well as fixing a plethora of issues reported since then.

nguyenluthuy249 commented 2 years ago

We are still using .net 5 and can't upgrade redis version without upgrading to .net 6. Is the latest version compatible with .net 5? Reference: https://www.nuget.org/packages/Microsoft.Extensions.Caching.StackExchangeRedis

mgravell commented 2 years ago

Yes. But the link you've provided: isn't "us". That's a completely different tool that sits on top of this library. If you have questions about that: that's a question for over there

nguyenluthuy249 commented 2 years ago

Yes, of course that isn't "you" but it loads "you" as the dependency. That is for dependency injection injecting your library and for .net 5. The issue occurs from StackExchange.Redis.

Get Outlook for Androidhttps://aka.ms/AAb9ysg


From: Marc Gravell @.> Sent: Tuesday, May 10, 2022 12:40:24 PM To: StackExchange/StackExchange.Redis @.> Cc: Thuy Nguyen @.>; Author @.> Subject: Re: [StackExchange/StackExchange.Redis] Random StackExchange.Redis.RedisTimeoutException (Issue #2137)

Yes. But the link you've provided: isn't "us". That's a completely different tool that sits on top of this library. If you have questions about that: that's a question for over there

— Reply to this email directly, view it on GitHubhttps://github.com/StackExchange/StackExchange.Redis/issues/2137#issuecomment-1122740569, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADSEYWOFB74SDBRLBGA57STVJKURRANCNFSM5VSLCSSQ. You are receiving this because you authored the thread.Message ID: @.***>

NickCraver commented 2 years ago

@nguyenluthuy249 You can explicitly load a version of us that's later/compatible (it remains compatible) beyond what their dependencies require. Just add an explicit include for StackExchange.Redis for the latest version.

chengang-hub commented 2 years ago

I am also getting TimeoutException:

StackExchange.Redis.RedisTimeoutException: Timeout performing GET (5000ms), next: GET Identity:Session:1j8fP7Ai2ESGgEWcKOMN+pkk, inst: 3, qu: 0, qs: 10, aw: False, rs: ReadAsync, ws: Idle, in: 8384, in-pipe: 0, out-pipe: 0, serverEndpoint: , mgr: 9 of 10 available, clientName: yingkehuiyuan-5db544c847-t9vll, IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=2963,Free=29804,Min=12402,Max=32767), v: 2.0.601.3402 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)

StackExchange.Redis.RedisTimeoutException: Timeout performing GET (5000ms), next: GET Identity:Session:1j8fP7Ai2ESGgEWcKOMN+pkk, inst: 3, qu: 0, qs: 10, aw: False, rs: ReadAsync, ws: Idle, in: 8384, in-pipe: 0, out-pipe: 0, serverEndpoint: \, mgr: 9 of 10 available, clientName: yingkehuiyuan-5db544c847-t9vll, IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=2980,Free=29787,Min=12402,Max=32767), v: 2.0.601.3402 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)

StackExchange.Redis.RedisTimeoutException: Timeout performing GET (5000ms), next: GET Identity:Session:1j8fP7Ai2ESGgEWcKOMN+pkk, inst: 3, qu: 0, qs: 10, aw: False, rs: ReadAsync, ws: Idle, in: 8384, in-pipe: 0, out-pipe: 0, serverEndpoint: mgr: 9 of 10 available, clientName: yingkehuiyuan-5db544c847-t9vll, IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=3017,Free=29750,Min=12402,Max=32767), v: 2.0.601.3402 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)

StackExchange.Redis.RedisTimeoutException: Timeout performing GET (5000ms), next: GET Identity:Session:1j8fP7Ai2ESGgEWcKOMN+pkk, inst: 3, qu: 0, qs: 10, aw: False, rs: ReadAsync, ws: Idle, in: 8384, in-pipe: 0, out-pipe: 0, serverEndpoint: , mgr: 9 of 10 available, clientName: yingkehuiyuan-5db544c847-t9vll, IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=2963,Free=29804,Min=12402,Max=32767), v: 2.0.601.3402 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)

StackExchange.Redis.RedisTimeoutException: Timeout performing GET (5000ms), active: GET Identity:Session:5/+Moz+Oz0uXO3T19EYe6cxq, next: GET Identity:Session:9PnUYOgWoUCasxxR5e8Jtkt3, inst: 4, qu: 0, qs: 2, aw: False, rs: CompletePendingMessage, ws: Idle, in: 0, in-pipe: 4086, out-pipe: 0, serverEndpoint: , mgr: 7 of 10 available, clientName: yingkehuiyuan-5db544c847-t9vll, IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=1482,Free=31285,Min=12402,Max=32767), v: 2.0.601.3402 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)