search

opserver / Opserver

Stack Exchange's Monitoring System
https://opserver.github.io/Opserver/
MIT License
4.52k stars 825 forks source link

Redis: "An invalid IP address was specified." for master node #411

Open movergan opened 2 years ago

movergan commented 2 years ago

I have the following config:

   "Redis": {
      "Servers": [
        {
          "name": "redis-node-0.redis-headless.d003.svc.cluster.local",
          "instances": [
            {
              "name": "Redis-0",
              "port": 6379,
              "password": "xxx"
            }
          ]
        },
        {
          "name": "redis-node-1.redis-headless.d003.svc.cluster.local",
          "instances": [
            {
              "name": "Redis-1",
              "port": 6379,
              "password": "xxx"
            }
          ]
        },
        {
          "name": "redis-node-2.redis-headless.d003.svc.cluster.local",
          "instances": [
            {
              "name": "Redis-2",
              "port": 6379,
              "password": "xxx"
            }
          ]
        }
      ]
    },

If I go to any replicas, I get correct information, when I go to Master, I get this 500 error:

System.FormatException: An invalid IP address was specified.
 ---> System.Net.Sockets.SocketException (22): Invalid argument
   --- End of inner exception stack trace ---
   at System.Net.IPAddressParser.Parse(ReadOnlySpan<T> ipSpan, Boolean tryParse)
   at System.Net.IPAddress.Parse(String ipString)
   at Opserver.Data.Redis.RedisInfo.RedisReplicaInfo.get_IPAddress() in /app/src/Opserver.Core/Data/Redis/RedisInfo.cs:line 157
   at Opserver.Data.Redis.RedisModule.GetInstance(RedisReplicaInfo info) in /app/src/Opserver.Core/Data/Redis/RedisModule.cs:line 139
   at Opserver.Data.Redis.RedisInstance.<get_ReplicaInstances>b__104_0(RedisReplicaInfo s) in /app/src/Opserver.Core/Data/Redis/RedisInstance.Info.cs:line 103
   at System.Linq.Enumerable.SelectListIterator<T1,T2>.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator<T>.ToList()
   at Opserver.Data.Redis.RedisInstance.get_ReplicaInstances() in /app/src/Opserver.Core/Data/Redis/RedisInstance.Info.cs:line 103
   at Opserver.Data.Redis.RedisInstance.GetAllReplicasInChain() in /app/src/Opserver.Core/Data/Redis/RedisInstance.Info.cs:line 111
   at AspNetCoreGeneratedDocument.Views_Redis_Dashboard_Instances.<>c.<SectionHeader>b__3_0(RedisInstance h) in /app/src/Opserver.Web/Views/Redis/Dashboard.Instances.cshtml:line 34
   at System.Linq.Enumerable.SelectManySingleSelectorIterator<T1,T2>.MoveNext()
   at System.Collections.Generic.HashSet<T>.UnionWith(IEnumerable<T> other)
   at System.Linq.Enumerable.UnionIterator<T>.FillSet()
   at System.Linq.Enumerable.UnionIterator<T>.ToList()
   at AspNetCoreGeneratedDocument.Views_Redis_Dashboard_Instances.SectionHeader(List<T> instances, String label, Boolean defaultHeaders, Boolean countChain) in /app/src/Opserver.Web/Views/Redis/Dashboard.Instances.cshtml:line 34
   at AspNetCoreGeneratedDocument.Views_Redis_Dashboard_Instances.ExecuteAsync() in /app/src/Opserver.Web/Views/Redis/Dashboard.Instances.cshtml:line 185
   at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage page, ViewContext context)
   at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage page, ViewContext context, Boolean invokeViewStarts)
   at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context)
   at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable<T> statusCode)
   at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable<T> statusCode)
   at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ActionContext actionContext, IView view, ViewDataDictionary viewData, ITempDataDictionary tempData, String contentType, Nullable<T> statusCode)
   at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewResultExecutor.ExecuteAsync(ActionContext context, ViewResult result)
   at Microsoft.AspNetCore.Mvc.ViewResult.ExecuteResultAsync(ActionContext context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeResultAsync>g__Logged|22_0(ResourceInvoker invoker, IActionResult result)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|30_0<TFilter,TFilterAsync>(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext<TFilter,TFilterAsync>(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.ResponseCaching.ResponseCachingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Authorization.Policy.AuthorizationMiddlewareResultHandler.HandleAsync(RequestDelegate next, HttpContext context, AuthorizationPolicy policy, PolicyAuthorizationResult authorizeResult)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at StackExchange.Profiling.MiniProfilerMiddleware.Invoke(HttpContext context) in C:\projects\dotnet\src\MiniProfiler.AspNetCore\MiniProfilerMiddleware.cs:line 121
   at StackExchange.Exceptional.ExceptionalMiddleware.Invoke(HttpContext context)

Full Trace:

   at StackExchange.Exceptional.ExceptionalMiddleware.Invoke(HttpContext context)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start<TStateMachine>(TStateMachine& stateMachine)
   at StackExchange.Exceptional.ExceptionalMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start<TStateMachine>(TStateMachine& stateMachine)
   at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)
   at Microsoft.AspNetCore.HostFiltering.HostFilteringMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests<TContext>(IHttpApplication<T> application)
   at async System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()

The same error is on the dashboard.

I use Redis server 1 master 2 replicas, installed via Bitnami helm chart in K8S. Image version is: "docker.io/bitnami/redis:6.2.6-debian-10-r146"

movergan commented 2 years ago

Looks like it fails to parse the following lines returned by Master:

slave0:ip=redis-node-0.redis-headless.d003.svc.cluster.local,port=6379,state=online,offset=11855465,lag=0
slave1:ip=redis-node-2.redis-headless.d003.svc.cluster.local,port=6379,state=online,offset=11855249,lag=1

Somewhere here: https://github.com/opserver/Opserver/blob/a47b1fc69b33fabbdeeed6b714e7afd17dcf544b/src/Opserver.Core/Data/Redis/RedisInfo.cs#L157