spotahome / redis-operator

Redis Operator creates/configures/manages high availability redis with sentinel automatic failover atop Kubernetes.
Apache License 2.0
1.5k stars 356 forks source link

Master distribution: pick master at random during first boot. #523

Closed samof76 closed 1 year ago

samof76 commented 1 year ago

image

The master distribution, is fairly skewed, and might be cause of worry when running multiple redis-failover deployments and many masters get crowded to a single node.

samof76 commented 1 year ago

Here is the query...

count(redis_instance_info{k8s_cluster_name="$cluster", redisfailovers_role="master"}) by (node)
samof76 commented 1 year ago

@ese and @Wouter0100 may must randomize the allotement instead of picking the zeroth pod.

ese commented 1 year ago

IMHO this depends a lot on the specific use case and the architecture where you are integrating your Redis clusters. From my point of view, different Redis clusters are independent so it is not relevant if they share the same node for their masters while they slaves are well distributed across failure domains. The schedule constraints should be managed by Kubernetes and the operator should look for mechanisms to leverage like topologySpreadConstraints, I think maybe a more reliable labeling of the pods to mark the Redis role of Redis instances could help on this task

Wouter0100 commented 1 year ago

I agree with @ese.

samof76 commented 1 year ago

Ok makes sense

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 45 days with no activity.

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 14 days since being marked as stale.