redis / go-redis

Redis Go client
https://redis.uptrace.dev
BSD 2-Clause "Simplified" License
19.89k stars 2.34k forks source link

Seeing "all sentinels specified in configuration are unreachable" error #2799

Open romansonrai opened 10 months ago

romansonrai commented 10 months ago

Description

I am seeing the following error returned from my redis client: "all sentinels specified in configuration are unreachable" It is happening relatively infrequently, but often enough to cause issues with the application code. I do not see any corresponding errors in the redis server logs.

Expected Behavior

In theory I should never see this error.

Current Behavior

I get "all sentinels specified in configuration are unreachable" periodically (maybe 1% of the time). Do I need to implement retry logic?

Steps to Reproduce

  1. I do not know how to reproduce

Context (Environment)

This error has caused the lookup of cached queries to fail, and redis Counters not to be reliable. Redis counters are used to bypass certain functionality in the app if a count reaches a certain threshold.

Version

github.com/redis/go-redis/v9 v9.3.0

Attempts to remediate

  1. set redis client configuration to include:
        client = redis.NewClient(&redis.Options{
            Addr:           config.Local.Redis.Addr,
            MaxRetries:     5,
            PoolSize:       200,
            MaxActiveConns: 0,
        })
  2. disable any other redis clusters (they all run on the same port)
  3. Test in 3 completely different environments
deanpatel2 commented 4 days ago

Having same issue when configuring Redis Sentinel with JuiceFS