Add support for persistentVolumeClaimRetentionPolicy on StatefulSets

[joebg10]

What is missing? An ability to choose whether k8s deletes or retains PVCs in tandem with the lifecycle of a StatefulSet, due to the new beta field spec.persistentVolumeClaimRetentionPolicy in the latest version of Kubernetes. There is no way to configure the StatefulSet of Prometheus in kube-prometheus.

Why do we need it? Because the new field defaults to Retain, PVCs remain bound to their PVs during cluster deletion while StatefulSets are being removed. This provides surface for a CSI Driver to see a stale, but intact PVC from a pod who no longer is requesting resources during cluster deletion. The CSI Driver then lacks the ability to manage(delete) unbound, available Volumes that otherwise would have been destroyed, resulting in orphaned resources.

[AliDev0ps]

Unfortunately the upstream prometheus-operator does not support this field

API Document:

• https://prometheus-operator.dev/docs/operator/api/#monitoring.coreos.com/v1.PrometheusSpec

Reference source code:

• https://github.com/prometheus-operator/prometheus-operator/blob/d86d6a449f716137acdbc6d0a2fbce4a7649bda3/pkg/apis/monitoring/v1/prometheus_types.go#L61-L600

[simonpasquier]

https://github.com/prometheus-operator/prometheus-operator/issues/1897 is the corresponding (and blocking) operator issue.

[ArthurSens]

Issue solved upstream, should be good to advance here once new release is out!