redis-operator not reachable webhook

[dntosas]

Does this issue reproduce with the latest release? Yes What operating system and processor architecture are you using (kubectl version)?

kubectl version Output

$ kubectl version

What did you do?

Error from server: conversion webhook for redis.redis.opstreelabs.in/v1beta1, Kind=RedisCluster failed: Post "https://XXX.svc:443/convert?timeout=30s": dial tcp XXX:443: connect: connection refused

checked that there are healthy endpoints on this service

What did you expect to see?

What did you see instead?

[jwitko]

It appears that the operator is forcing the service lookup to a hard-coded hostname as best I can tell. Can't exactly tell where thats happening

[jwitko]

I see it now, its in the CRD: https://github.com/OT-CONTAINER-KIT/helm-charts/blob/main/charts/redis-operator/crds/redis-replication.yaml#L7719C1-L7727C25

[shubham-cmyk]

@jwitko

Do you mean CA injection?

[jwitko]

@jwitko

Do you mean CA injection?

No, The namespace for the webhook service is directly hard-coded into the CRDs. Linked above.

[jwitko]

For example: When I deploy redis-replication CRD I get:

conversion webhook for redis.redis.opstreelabs.in/v1beta1, Kind=RedisReplication failed: Post "https://webhook-service.redis-operator.svc:443/convert?timeout=30s": service "webhook-service" not found

If I remove the following section from the CRD:

  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        service:
          name: webhook-service
          namespace: redis-operator
          path: /convert
          port: 443
      conversionReviewVersions:
        - v1beta1
        - v1beta2

It works without issue.

[praveenperera]

I'm having the same issue, fixed by downgrading to 0.15.3

[Routhinator]

Confirming that 0.15.3 is the last working version of the chart, but also a note to others: You must delete the CRDs before downgrade for it to be successful.

[enigma-kazbek]

Downgrade works for me:

kubectl delete crd $(kubectl get crd | grep redis.opstreelabs.in | awk '{print $1}')
helm -n redis-operator un redis-operator
helm upgrade --install redis-operator ot-helm/redis-operator -n redis-operator --version 0.15.3

---
apiVersion: redis.redis.opstreelabs.in/v1beta1
kind: Redis
metadata:
  name: redis-standalone
  namespace: bcs-dev
spec:
  kubernetesConfig:
    image: quay.io/opstree/redis:v7.0.5
    imagePullPolicy: IfNotPresent
  storage:
    volumeClaimTemplate:
      spec:
        # storageClassName: standard
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi
  securityContext:
    runAsUser: 1001

[borg286]

I fixed this by hard-coding the namespace where I deploy the redis-operator into so the namespace is "redis-operator" Then the hard-coded url it uses ends up finding it at the right place.

The issue I'm facing after that is that the pod that is the operator itself doesn't allocate port 9443. Thus I end up getting http error code 503 when trying to hit it.

[safehome-jdev]

I see it now, its in the CRD: https://github.com/OT-CONTAINER-KIT/helm-charts/blob/main/charts/redis-operator/crds/redis-replication.yaml#L7719C1-L7727C25

As @jwitko states, the namespace is hard-coded into the CRD, so if you move the operator anywhere besides a namespace of redis-operator, it breaks the deployment. This definitely needs a fix.

• redis
• redis-sentinel
• redis-replication
• redis-cluster