Operator tries to update STS forbidden fields

[ant31]

Hi, The reconcilation is stuck because it tries to update the statefulset with forbidden fields. I'm not sure exactly what the operator is trying to reconcile, I guess it's the Master IP but I'm not sure. When I deleted the sts to unblock the operator the diff between old sts and new sts was:

          - name: MASTER_GROUP_NAME
            value: myMaster
          - name: IP
!           value: 10.90.1.201
          - name: PORT
            value: "6379"
          - name: QUORUM
--- 72,78 ----
          - name: MASTER_GROUP_NAME
            value: myMaster
          - name: IP
!           value: 10.90.1.173
          - name: PORT
            value: "6379"
          - name: QUORUM

What version of redis operator are you using? 0.15.1

{"level":"error","ts":"2024-01-22T15:16:07Z",
"logger":"controllers.RedisSentinel",
"msg":"Cannot create Sentinel statefulset for Redis",
"error":"StatefulSet.apps \"redis-sentinel-engine-sentinel\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than replicas, ordinals, template, updateStrategy, persistentVolumeClaimRetentionPolicy and minReadySeconds are forbidden",
"stacktrace":"github.com/OT-CONTAINER-KIT/redis-operator/k8sutils.RedisSentinelSTS.CreateRedisSentinelSetup
        /workspace/k8sutils/redis-sentinel.go:85
github.com/OT-CONTAINER-KIT/redis-operator/k8sutils.CreateRedisSentinel
        /workspace/k8sutils/redis-sentinel.go:54
github.com/OT-CONTAINER-KIT/redis-operator/controllers.(*RedisSentinelReconciler).Reconcile
        /workspace/controllers/redissentinel_controller.go:58
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:119
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:316
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.2/pkg/internal/controller/controller.go:227"}

Does this issue reproduce with the latest release? yes

[drivebyer]

IMO, it might not be the Master IP, because it's in the StatefulSet spec's template field. According to the error message:

"error": "StatefulSet.apps 'redis-sentinel-engine-sentinel' is invalid: spec: Forbidden: updates to StatefulSet spec for fields other than 'replicas', 'ordinal', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy', and 'minReadySeconds' are forbidden."

It can be modified.

[jaradhuggard]

Hello, I'm having the same issue with this. I've looked at the patch the operator is trying to apply.

{
  "metadata": {
    "annotations": {
      "linkerd.io/inject": "disabled"
    },
    "labels": {
      "helm.sh/chart": "redis-sentinel-0.15.7",
      "tanka.dev/environment": null
    }
  },
  "spec": {
    "selector": {
      "matchLabels": {
        "helm.sh/chart": "redis-sentinel-0.15.7",
        "tanka.dev/environment": null
      }
    },
    "template": {
      "metadata": {
        "annotations": {
          "linkerd.io/inject": "disabled"
        },
        "labels": {
          "helm.sh/chart": "redis-sentinel-0.15.7",
          "tanka.dev/environment": null
        }
      },
      "spec": {
        "$setElementOrder/containers": [
          {
            "name": "redis-sentinel-sentinel"
          }
        ],
        "containers": [
          {
            "$setElementOrder/env": [
              {
                "name": "REDIS_ADDR"
              },
              {
                "name": "SERVER_MODE"
              },
              {
                "name": "SETUP_MODE"
              },
              {
                "name": "MASTER_GROUP_NAME"
              },
              {
                "name": "IP"
              },
              {
                "name": "PORT"
              },
              {
                "name": "QUORUM"
              },
              {
                "name": "DOWN_AFTER_MILLISECONDS"
              },
              {
                "name": "PARALLEL_SYNCS"
              },
              {
                "name": "FAILOVER_TIMEOUT"
              }
            ],
            "env": [
              {
                "name": "IP",
                "value": null
              }
            ],
            "name": "redis-sentinel-sentinel"
          }
        ]
      }
    }
  }
}

The only things that could be issues, when looking at the patch, are the spec.selector.matchLabels and metadata.labels. Am I right in thinking that metadata should be okay as well? If so, would be an issue of ensuring that spec.selector.matchLabels is remaining the same value?

[ant31]

Yes, it's probably the matchLabels

 * spec: Forbidden: updates to statefulset spec for fields other than:
  'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy'
  and 'minReadySeconds' are forbidden

when trying to update spec.selector.matchLabels manually (via kubectl edit)