OT-CONTAINER-KIT / redis-operator

A golang based redis operator that will make/oversee Redis standalone/cluster/replication/sentinel mode setup on top of the Kubernetes.
https://ot-redis-operator.netlify.app/
Apache License 2.0
731 stars 206 forks source link

Replication master host is unreachable after data cleanup #981

Open alita1991 opened 2 weeks ago

alita1991 commented 2 weeks ago

What version of redis operator are you using?

kubectl logs <_redis-operator_pod_name> -n <namespace>

redis-operator version: 0.16.0

Does this issue reproduce with the latest release? Yes

What operating system and processor architecture are you using (kubectl version)?

kubectl version Output
$ kubectl version
Client Version: v1.29.3

What did you do?

  1. Install redis replicated + sentinel v7.2.3
  2. Downgrade replicated + sentinel to version v6.2.14 (with data cleanup via init container + appendonly: no, save "")
  3. Reconfigure replication + sentinel and remove appendonly: no + save "" and init container

What did you expect to see?

Replication slaves should be connected with the master

What did you see instead?

1:S 11 Jun 2024 14:54:03.933 # Error condition on socket for SYNC: Host is unreachable
1:S 11 Jun 2024 14:54:04.870 * Connecting to MASTER 10.42.2.219:6379
1:S 11 Jun 2024 14:54:04.870 * MASTER <-> REPLICA sync started
1:S 11 Jun 2024 14:54:07.933 # Error condition on socket for SYNC: Host is unreachable
1:S 11 Jun 2024 14:54:08.883 * Connecting to MASTER 10.42.2.219:6379
1:S 11 Jun 2024 14:54:08.883 * MASTER <-> REPLICA sync started
1:S 11 Jun 2024 14:54:11.933 # Error condition on socket for SYNC: Host is unreachable
1:S 11 Jun 2024 14:54:12.896 * Connecting to MASTER 10.42.2.219:6379
1:S 11 Jun 2024 14:54:12.896 * MASTER <-> REPLICA sync started

Workaround After 30 min, I deleted the replication pods 2 times and I was able to recover the cluster.

drivebyer commented 1 week ago

Please try it with v0.17.0 @alita1991