OT-CONTAINER-KIT / redis-operator

A golang based redis operator that will make/oversee Redis standalone/cluster/replication/sentinel mode setup on top of the Kubernetes.
https://ot-redis-operator.netlify.app/
Apache License 2.0
734 stars 207 forks source link

RedisCluster not up and running #1015

Open joanalcarneiro opened 2 days ago

joanalcarneiro commented 2 days ago

Thanks a lot for building this operator! :) I am facing some issues and I am kindly requesting for your help :)

Does this issue reproduce with the latest release?

yes --> using release 0.17.0

helm repo add ot-helm https://ot-container-kit.github.io/helm-charts/

helm install redis-operator ot-helm/redis-operator --create-namespace --namespace ot-operators

helm test redis-operator --namespace ot-operators

What operating system and processor architecture are you using (kubectl version)?

kubectl version Output
$ kubectl version
Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.0

What did you do?

kubectl create namespace redis-cluster
kubectl apply -f custom-resource-definitions/redis-cluster-basic.yaml -n redis-cluster

where redis-cluster-basic.yaml is

apiVersion: redis.redis.opstreelabs.in/v1beta2
kind: RedisCluster
metadata:
  name: redis-cluster
spec:
  clusterSize: 3 # cluster node count - 1 shard with 3 nodes
  clusterVersion: v6
  persistenceEnabled: false
  redisLeader:
    replicas: 1 # number of nodes that are primary nodes
    readinessProbe: # we need it in order for the operator not to crash
      initialDelaySeconds: 5
      periodSeconds: 5
      timeoutSeconds: 1
      successThreshold: 1
      failureThreshold: 3
    livenessProbe: # we need it in order for the operator not to crash
      initialDelaySeconds: 5
      periodSeconds: 5
      timeoutSeconds: 1
      successThreshold: 1
      failureThreshold: 3
  redisFollower:
    replicas: 2 # number of nodes that are read nodes
    readinessProbe: # we need it in order for the operator not to crash
      initialDelaySeconds: 5
      periodSeconds: 5
      timeoutSeconds: 1
      successThreshold: 1
      failureThreshold: 3
    livenessProbe: # we need it in order for the operator not to crash
      initialDelaySeconds: 5
      periodSeconds: 5
      timeoutSeconds: 1
      successThreshold: 1
      failureThreshold: 3
  kubernetesConfig:
    image: quay.io/opstree/redis:v6.2.14
    imagePullPolicy: IfNotPresent
    resources:
      limits:
        cpu: 1
        memory: 2Gi
  storage: # we need it in order to keep node.conf at least
    volumeClaimTemplate:
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi
    nodeConfVolume: true
    nodeConfVolumeClaimTemplate:
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi

What did you expect to see?

What did you see instead?

I see 1 leader and 2 followers created with success and no errors.

But when

But when checking the cluster I see that the leader node has no knowledge of the follower nodes.


kubectl exec -it redis-cluster-leader-0 -n redis-cluster -- /bin/bash                                               ✔ │ kind-kind ⎈ │ 16:38:10  ▓▒░
bash-5.1$ redis-cli -c
127.0.0.1:6379> CLUSTER INFO
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0
127.0.0.1:6379> CLUSTER NODES
a12225caf10733a77e299ee5dea79fd1be99b7ca 10.244.1.14:6379@16379 myself,master - 0 0 0 connected
drivebyer commented 1 day ago

@joanalcarneiro, if you want to set one leader with two replicas in each set, you should set spec.redisLeader.replicas=3 and spec.redisFollower.replicas=6.