Curious: I'm seeing this on Kafka cluster tool (Strimzi), they provide a way to not rely on the "pod disruption budget" to make sure 2/3 instances are always up.
What I mean by this is that usually:
Kubernetes will do the rollout of pods
It adds a new pod and once ready, it removes an old one
But this does not take in account the time needed for all instances to sync between each other
There is a chance where Kubernetes will have for sure 2/3 pods "ready", but only "1/3" synchronous compared to before the rollout. Worst in case the rollout is really speed (they all become ready fast, without the time to sync with the last instances that had data before being drained).
I didn't face this issue your Redis operator but what I understand it's possible (so just asking and pointing this). Just wanted to know what you think about this, and if you already thought about this :) ?
From https://github.com/amaizfinance/redis-operator/issues/43:
Hi,
Curious: I'm seeing this on Kafka cluster tool (Strimzi), they provide a way to not rely on the "pod disruption budget" to make sure 2/3 instances are always up.
What I mean by this is that usually:
There is a chance where Kubernetes will have for sure 2/3 pods "ready", but only "1/3" synchronous compared to before the rollout. Worst in case the rollout is really speed (they all become ready fast, without the time to sync with the last instances that had data before being drained).
Here their tool: https://github.com/strimzi/drain-cleaner
I didn't face this issue your Redis operator but what I understand it's possible (so just asking and pointing this). Just wanted to know what you think about this, and if you already thought about this :) ?
Thank you,
cc @nrvnrvn