projectcalico / calico

Cloud native networking and network security
https://docs.tigera.io/calico/latest/about/
Apache License 2.0
5.88k stars 1.31k forks source link

Calico upgrade through operator doesn't remove deprecated k8s resources. #9180

Closed scardena closed 4 days ago

scardena commented 2 weeks ago

Expected Behavior

PSPs are removed from k8s 1.24 to 1.25. I upgraded calico from 3.25.0 to 3.28.1 using the operator by running:

 kubectl apply --server-side --force-conflicts -f https://raw.githubusercontent.com/projectcalico/calico/v3.28.1/manifests/tigera-operator.yaml

in my k8s 1.24 cluster. I was expecting the deprecated/retired resources/objects (PSPs) to get deleted automatically as they are no longer supported in new versions, but the resources are still there.

Current Behavior

Resources are still there:

kubectl get psp | grep calico
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
calico-kube-controllers      false          RunAsAny   MustRunAsNonRoot   MustRunAs   MustRunAs   false            configMap,downwardAPI,emptyDir,persistentVolumeClaim,projected,secret
calico-node                  true           RunAsAny   RunAsAny           MustRunAs   MustRunAs   false            configMap,downwardAPI,emptyDir,persistentVolumeClaim,projected,secret,hostPath
calico-typha                 false          RunAsAny   MustRunAsNonRoot   MustRunAs   MustRunAs   false            configMap,downwardAPI,emptyDir,persistentVolumeClaim,projected,secret

It seems the controller did get updated correctly to the latest image, so I am indeed in the latest calico version:

kubectl get pods -n calico-system -o yaml calico-node-srsql | grep image -i
    image: docker.io/calico/node:v3.28.1

Possible Solution

Manually removing all those resources? I am not sure what can happen and I don't want to leave some other cruft dangling.

Your Environment

lwr20 commented 2 weeks ago

IIUC, tigera-operator creates PSPs only if the cluster its installed on supports them. It looks like while they are deprecated in your cluster, they still work, so operator maintains them to provide maximum compatibility.

I upgraded calico from 3.25.0 to 3.28.1 using the operator by running: ... in my k8s 1.24 cluster

As per https://docs.tigera.io/calico/3.27/getting-started/kubernetes/requirements#kubernetes-requirements, Calico v3.28.1 does not support kubernetes 1.24. You need to upgrade that.

In fact, it looks like the PSP support detection (and PSPs in general) were removed entirely in https://github.com/tigera/operator/pull/3374, because newer versions of operator do not support versions of kubernetes that have PSPs.