expose `persistentVolumeClaimRetentionPolicy` in StarRocks operator CRD

[mingmxu]

Describe the current behavior

Currently it uses the default behaviour, Retain for both whenDeleted and whenScaled. It leaves PVC un-used when a cluster is deleted or scaled.

Describe the enhancement

By exposing persistentVolumeClaimRetentionPolicy, user could specify different policies explicitly. For example Retain for production cluster, Delete for test/ephemeral clusters.

Additional context

[yandongxiao]

    persistentVolumeClaimRetentionPolicy describes the lifecycle of persistent
    volume claims created from volumeClaimTemplates. By default, all persistent
    volume claims are created as needed and retained until manually deleted.
    This policy allows the lifecycle to be altered, for example by deleting
    persistent volume claims when their stateful set is deleted, or when their
    pod is scaled down. This requires the StatefulSetAutoDeletePVC feature gate
    to be enabled, which is alpha.  +optional
    StatefulSetPersistentVolumeClaimRetentionPolicy describes the policy used
    for PVCs created from the StatefulSet VolumeClaimTemplates.

StatefulSetAutoDeletePVC is in alpha state, and my k8s version is v1.27.3. So, Event we expose it, It may not take effect on user's k8s environment.

[zy2xu]

I have the same problem. Is there a workaround other than manually delete?

[yandongxiao]

‌‌‌Besides manual deletion, there is no other way at present. Directly exposing the persistentVolumeClaimRetentionPolicy field of the StatefulSet may lead to non-effectiveness of the field due to different users' Kubernetes versions. I think it can be accomplished by the Operator: provide a option (for example, named persistentVolumeClaimRetentionPolicy), which allows the Operator to be responsible for deleting the corresponding PVC after the StarRocksCluster is deleted.