groundhog2k / helm-charts

Helm charts for open source applications - ready to use for deployment on Kubernetes
MIT License
69 stars 56 forks source link

Improve logging and failover detection during Redis-HA initialization #1193

Closed groundhog2k closed 1 year ago

groundhog2k commented 1 year ago

Improve the logging output for the init container during Redis-HA startup. Implement additional DNS lookup to make sure the detected master instance is still valid.

Additional: Implement support for building a Redis HA cluster using FQDN DNS names instead of IP addresses.

Background:

When a master instance is terminated the Redis HA cluster still assumes the instance is a valid master - it takes time for the cluster to find out that a cluster instance is down (see downAfterMilliseconds:). The DNS lookup adds an additonal "safety layer". When the DNS lookup fails the init will wait (see failoverWait:) until another attempt to detect the master is tried. If second attempt fails again, a failover will be manually forced.