bjw-s / helm-charts

A collection of Helm charts
https://bjw-s.github.io/helm-charts/
Apache License 2.0
535 stars 100 forks source link

Upgrade 1.x - >2.x, can't becuase labelselector is immutable #195

Closed evanrich closed 9 months ago

evanrich commented 9 months ago

Details

What steps did you take and what happened:

Tried to upgrade 1.5.1 -> 2.x, followed the values.yaml changes as best i could. Ran through argoCD to update, it fails to deploy because of an immutable field

exact error:

Deployment.apps "sonarr" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/component":"main", "app.kubernetes.io/instance":"sonarr", "app.kubernetes.io/name":"sonarr"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

1.5.1 format:

image:
  repository: ghcr.io/onedr0p/sonarr-develop
  pullPolicy: IfNotPresent
  tag: 4.0.0.688
strategy:
  type: Recreate
podSecurityContext:
  runAsUser: 1000
  runAsGroup: 1000
  fsGroup: 1000
  fsGroupChangePolicy: "OnRootMismatch"
env:
  TZ: America/Los_Angeles
  SONARR__API_KEY:
    valueFrom:
      secretKeyRef:
        name: sonarr-api-key
        key: sonarr_api_key
service:
  main:
    ports:
      http:
        port: 8989
ingress:
  main:
    enabled: true
    ingressClassName: traefik
    hosts:
      - host: server.domain.tld
        paths:
          - path: /
            pathType: Prefix
            service:
              port: 8989
persistence:
  config:
    enabled: true
    type: pvc
    accessMode: ReadWriteOnce
    size: 15Gi
    mountPath: "/config"
  media:
    enabled: true
    type: nfs
    server: server.domain.tld
    path: /mnt/sirius/media
    mountPath: "/media-nfs"
  backups:
    enabled: true
    type: nfs
    server: server.domain.tld
    path: /mnt/sirius/dockerdata/backups/sonarr
    mountPath: "/config/Backups"
  dockerdata:
    enabled: true
    type: nfs
    server: server.domain.tld
    path: /mnt/sirius/dockerdata/
    mountPath: "/mnt/dockerdata"
probes:
  liveness:
    enabled: true
    custom: true
    spec:
      httpGet:
        path: /
        port: http
resources:
  requests:
    cpu: 300m
    memory: 768Mi
  limits:
    memory: 3Gi

2.x format:

defaultPodOptions:
  securityContext:
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 1000
    fsGroupChangePolicy: "OnRootMismatch"
controllers:
  main:
    containers:
      main:
        strategy:
          type: Recreate
        image:
          repository: ghcr.io/onedr0p/sonarr-develop
          pullPolicy: IfNotPresent
          tag: 4.0.0.688
        probes:
          liveness:
            enabled: true
            custom: true
            spec:
              httpGet:
                path: /
                port: http
        env:
          TZ: America/Los_Angeles
          SONARR__API_KEY:
            valueFrom:
              secretKeyRef:
                name: sonarr-api-key
                key: sonarr_api_key
        resources:
          requests:
            cpu: 300m
            memory: 768Mi
          limits:
            memory: 3Gi
service:
  main:
    ports:
      http:
        port: 8989

ingress:
  main:
    enabled: true
    className: traefik
    hosts:
      - host: server.domain.tld
        paths:
          - path: /
            pathType: Prefix
            service:
              name: main
              port: 8989
persistence:
  config:
    enabled: true
    type: persistentVolumeClaim
    accessMode: ReadWriteOnce
    size: 15Gi
    globalMounts:
      - path: "/config"
  media:
    enabled: true
    type: nfs
    server: server.domain.tld
    path: /mnt/sirius/media
    globalMounts:
      - path: "/media-nfs"
  backups:
    enabled: true
    type: nfs
    server: server.domain.tld
    path: /mnt/sirius/dockerdata/backups/sonarr
    globalMounts:
      - path: "/config/Backups"
  dockerdata:
    enabled: true
    type: nfs
    server: server.domain.tld
    path: /mnt/sirius/dockerdata/
    globalMounts:
      - path: "/mnt/dockerdata"

am I missing something here? the values seem to lint fine to ArgoCD. thanks! FWIW, this is what ArgoCD shows as the deltas: image

And the section in particular: image

bjw-s commented 9 months ago

Hey, thanks for creating this issue! Unfortunately there is nothing that I can do about that. Since Kubernetes makes the labelSelector immutable an in-place upgrade of Deployments unfortunately isn't possible. I will add this to the upgrade notes!

For some more backround: https://www.datree.io/resources/kubernetes-error-codes-field-is-immutable

evanrich commented 9 months ago

FWIW, I was able to work around this using ArgoCD's options: Force, Skip Schema Validation, and Replace image

this combination basically replaced the chart with the new one. to be safe re: my PVC, i set it's retention policy to "Retain" first so it wouldn't delete in case of the forced replacement, the container came up with the new chart, and everything else persisted. Leaving this in case anyone else runs into the same issue.

Thanks!