redis / go-redis

Redis Go client
https://redis.uptrace.dev
BSD 2-Clause "Simplified" License
20.17k stars 2.37k forks source link

Make MASTERDOWN a retriable error in RedisCluster client #3164

Open justinmir opened 1 month ago

justinmir commented 1 month ago

When clusters are running with replica-server-stale-data no, replicas will return a MASTERDOWN error under two conditions:

  1. The primary has failed and we are not serving requests.
  2. A replica has just started and has not yet synced from the primary.

The former, primary has failed and we are not serving requests, is similar to a CLUSTERDOWN error and should be similarly retriable.

When a replica has just started and has not yet synced from the primary the request should be retried on other available nodes in the shard. Otherwise a percentage of the read requests to the shard will fail.

Examples when replica-server-stale-data no is enabled:

  1. In a cluster using ReadOnly with a single read replica, every read request will return errors to the client because MASTERDOWN is not a retriable error.
  2. In a cluster using RouteRandomly a percentage of the requests will return errors to the client based on if this server was selected.