NickCraver
commented
6 months ago The problem you're having is thread pool starvation somewhere, indicated by:
WORKER: (Busy=196,Free=32571,Min=400,Max=32767),
POOL: (Threads=420,QueuedItems=57,CompletedItems=486767659,Timers=58),I see you're all async against Redis so this is likely somewhere else in the application. I'd suggest taking a memory dump under load to see what all those threads are doing.
Hey team!
I am using google cloud.
Sometimes an error occurs when there are a lot of connection requests.
cpu is at 30% Redis usage wasn't much.
Can you help me with this?
I was a node js developer and I don't know much about the .net core ecosystem.
I'm sorry I'm not good at English.
[ERROR]|Microsoft.AspNetCore.Server.Kestrel|Connection id "0HN23RLK7FKE4", Request id "0HN23RLK7FKE4:000000E0": An unhandled exception was thrown by the application. StackExchange.Redis.RedisTimeoutException: Timeout awaiting response (outbound=26KiB, inbound=38KiB, 5016ms elapsed, timeout is 5000ms), command=EXPIRE, next: HGETALL session:some:info:10108995606, inst: 0, qu: 166, qs: 446, aw: True, bw: CheckingForTimeoutComplete, rs: DequeueResult, ws: Writing, in: 0, in-pipe: 77194, out-pipe: 0, last-in: 0, cur-in: 35572, sync-ops: 0, async-ops: 38340728, serverEndpoint: x.x.x.x:6379, conn-sec: 71086.43, aoc: 0, mc: 1/1/0, mgr: 9 of 10 available, clientName: dev01, IOCP: (Busy=0,Free=1000,Min=500,Max=1000), WORKER: (Busy=196,Free=32571,Min=400,Max=32767), POOL: (Threads=420,QueuedItems=57,CompletedItems=486767659,Timers=58), v: 2.7.33.41805 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts) at alchemist_server.Manager.RedisManager.KeyExpire(REDIS_DB_TYPE redisType, String key, Int32 expireSec) in /src/Manager/RedisManager.cs:line 271 at alchemist_server.Service.SessionService.SessionValidation(HttpContext context) in /src/Service/SessionService.cs:line 361 at alchemist_server.Middleware.SessionVerification.InvokeAsync(HttpContext context, IRedisManager redisManager, ISessionService sessionService) in /src/Middleware/SessionVerification.cs:line 47 at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests [TContext](IHttpApplication`1 application)