Open gramotei opened 6 days ago
We suddenly started experiencing timeouts in production.
Stack:
Azure App Service - P2v3
Azure Redis Cache - P1 (tried with Redis on VM, same result)
.NET (asp.net) - 6.0.28
Microsoft.Extensions.Caching.StackExchangeRedis - 6.0.28
CPU load like - 2% without spikes
Network like - 30/40 connections/sec without spikes also
There are no slow queries logged
Explored all options, read many articles.
Timeout awaiting response (outbound=0KiB, inbound=0KiB, 15249ms elapsed, timeout is 15000ms), command=HMGET, next: HMGET PROD:Tenant:XXXXXXXXX, inst: 0, qu: 0, qs: 0, aw: False, bw: SpinningDown, rs: ReadAsync, ws: Idle, in: 0, last-in: 0, cur-in: 0, sync-ops: 42, async-ops: 117823, serverEndpoint: xxxxxxx.redis.cache.windows.net:6380, conn-sec: 2107.98, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: xxxxxxxxxx_c3f6697074(SE.Redis-v2.8.0.27420), IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=1,Free=32766,Min=4,Max=32767), POOL: (Threads=8,QueuedItems=0,CompletedItems=2910368,Timers=27), v: 2.8.0.27420 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)
It would be great to have at least direction how to troubleshoot because now obvious reason for that
Do you have any idea what the size of the response is on the HMGET? We agree this looks off, 15 seconds is a very long time.
HMGET
We suddenly started experiencing timeouts in production.
Stack:
Azure App Service - P2v3
Azure Redis Cache - P1 (tried with Redis on VM, same result)
.NET (asp.net) - 6.0.28
Microsoft.Extensions.Caching.StackExchangeRedis - 6.0.28
CPU load like - 2% without spikes
Network like - 30/40 connections/sec without spikes also
There are no slow queries logged
Explored all options, read many articles.
It would be great to have at least direction how to troubleshoot because now obvious reason for that