2881099 / FreeRedis

🦄 FreeRedis is .NET40+ redis client. supports cluster, sentinel, master-slave, pub-sub, lua, pipeline, transaction, streams, client-side-caching, and pooling.
MIT License
911 stars 160 forks source link

阿里云主从redis出现链接超时,但是监控显示一切正常 #197

Open QianYS opened 1 month ago

QianYS commented 1 month ago

错误所示: 2024-07-31 21:23:42.156 +08:00 [ERR] Connection id "0HN57R8AORGCD", Request id "0HN57R8AORGCD:00000001": An unhandled exception was thrown by the application. System.Exception: 【xsusercenter.redis.rds.aliyuncs.com/18】Status unavailable, waiting for recovery. Connect to redis-server(xsusercenter.redis.rds.aliyuncs.com:6379 -> Unspecified/xsusercenter.redis.rds.aliyuncs.com:6379) timeout, DEBUG: Dns.GetHostEntry(xsusercenter.redis.rds.aliyuncs.com)=System.Net.IPHostEntry ---> System.TimeoutException: Connect to redis-server(xsusercenter.redis.rds.aliyuncs.com:6379 -> Unspecified/xsusercenter.redis.rds.aliyuncs.com:6379) timeout, DEBUG: Dns.GetHostEntry(xsusercenter.redis.rds.aliyuncs.com)=System.Net.IPHostEntry at FreeRedis.Internal.DefaultRedisSocket.Connect() at FreeRedis.Internal.DefaultRedisSocket.Write(CommandPacket cmd) at FreeRedis.RedisClient.SingleInsideAdapter.<>cDisplayClass5_01.<AdapterCall>b__0() at FreeRedis.RedisClient.LogCallCtrl[T](CommandPacket cmd, Func1 func, Boolean aopBefore, Boolean aopAfter) at FreeRedis.RedisClient.LogCall[T](CommandPacket cmd, Func1 func) at FreeRedis.RedisClient.SingleInsideAdapter.AdapterCall[TValue](CommandPacket cmd, Func2 parse) at FreeRedis.RedisClient.Call(CommandPacket cmd) at FreeRedis.Internal.RedisClientPoolPolicy.PrevReheatConnectionPool(ObjectPool1 pool, Int32 minPoolSize) --- End of inner exception stack trace --- at FreeRedis.Internal.ObjectPool.ObjectPool1.GetFree(Boolean checkAvailable) at FreeRedis.Internal.ObjectPool.ObjectPool1.Get(Nullable1 timeout) at FreeRedis.RedisClient.PoolingAdapter.GetRedisSocket(CommandPacket cmd) at FreeRedis.RedisClient.PoolingAdapter.<>c__DisplayClass10_0`1.<b0>d.MoveNext() --- End of stack trace from previous location --- at FreeRedis.RedisClient.LogCallAsync[T](CommandPacket cmd, Func1 func) at InternalApi.Controller.PhoneController.GetRedis(String redisConnStr, String num) in /app/InternalApi/Controller/PhoneController.cs:line 41 at InternalApi.Controller.PhoneController.QueryPhone(String phone) in /app/InternalApi/Controller/PhoneController.cs:line 74 at lambda_method24(Closure, Object) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask1 actionResultValueTask) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.gAwaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.gLogged|17_1(ResourceInvoker invoker) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker) at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|7_0(Endpoint endpoint, Task requestTask, ILogger logger) at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext) at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

发生异常时的阿里云监控均是正常 image image image 亦未找到慢读取情况

问题发生时间大约是7月31日21点-21点40,问题发生后短时间无法解决,故将redis读取调整为读取数据库 具体api代码如图 image

是否能有更具体的相关信息提现是因为什么而导致的连接超时,或者出现问题时应该如何去定位这个问题的根本原因

2881099 commented 1 month ago

怎么不用单例啊?

MoondanceZ commented 1 month ago

我也出现这种情况, 一般都是在站点发布之后出现, 很奇怪

2881099 commented 1 month ago

确保不是被redis防火墙拒绝,min pool size设置为1,避免预热导致防火墙错误判断为攻击而拒绝。

请确保网络环境通畅,健康。

MoondanceZ commented 1 month ago

min pool size 和 max pool size 怎么理解呢