Reconcile storagepolicyquota in syncer

[akankshapanse]

What this PR does / why we need it: This PR adds reconciler for StoragePolicyQuota objects in syncer, which looks for add/delete events of StoragePolicyQuota CRs and accordingly creates StoragePolicyUsage CRs. Earlier this processing was managed by simple informer event in syncer (with handlers policyQuotaCRAdded() & policyQuotaCRDeleted()), however if these informer events get missed, the creation of StoragePolicyUsage CR does not happen again (given that full sync creates StoragePolicyUsage CR if the namespace has any bound/pending volumes, but if namespace does not have any PVC, no StoragePolicyUsage CR gets created in full sync today) Absense of StoragePolicyUsage CR blocks all further PVC/Snapshot creation/update. Hence it is necessary to reconcile/retry these event handling if missed/failed any time before.

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Testing done: Testing in progress. Will test the failure case or event miss case specifically and update logs

Tested basic storagepolicyquota add scenario on WCP setup

root@421ad0245402d86ac44fc4ad5816ff91 [ ~ ]# kubectl get storagepolicyquota -n svc-tkg-domain-c10
NAME                                                          AGE
management-storage-policy-encryption-storagepolicyquota       12d
management-storage-policy-regular-storagepolicyquota          12d
management-storage-policy-single-node-storagepolicyquota      12d
management-storage-policy-stretched-lite-storagepolicyquota   12d
management-storage-policy-thin-storagepolicyquota             12d
vm-encryption-policy-storagepolicyquota                       12d
vsan-default-storage-policy-storagepolicyquota                12d
wcpglobal-storage-profile-storagepolicyquota                  12d
wcplocal-storage-profile-storagepolicyquota                   27h   <<< storagepolicyquota for policy wcplocal-storage-profile
root@421ad0245402d86ac44fc4ad5816ff91 [ ~ ]#

root@421ad0245402d86ac44fc4ad5816ff91 [ ~ ]#
root@421ad0245402d86ac44fc4ad5816ff91 [ ~ ]# kubectl get storagepolicyusage -n svc-tkg-domain-c10
NAME                                                 AGE
management-storage-policy-encryption-pvc-usage       12d
management-storage-policy-regular-pvc-usage          12d
management-storage-policy-single-node-pvc-usage      12d
management-storage-policy-stretched-lite-pvc-usage   12d
management-storage-policy-thin-pvc-usage             12d
vm-encryption-policy-pvc-usage                       12d
vsan-default-storage-policy-pvc-usage                12d
wcpglobal-storage-profile-pvc-usage                  12d             <<< manually deleted storagepolicyusage for policy "wcplocal-storage-profile"
root@421ad0245402d86ac44fc4ad5816ff91 [ ~ ]#

Updated the change in syncer:

root@421ad0245402d86ac44fc4ad5816ff91 [ ~ ]# kubectl edit deploy -n vmware-system-csi vsphere-csi-controller
deployment.apps/vsphere-csi-controller edited

root@421ad0245402d86ac44fc4ad5816ff91 [ ~ ]# kubectl get storagepolicyusage -n svc-tkg-domain-c10
NAME                                                 AGE
management-storage-policy-encryption-pvc-usage       12d
management-storage-policy-regular-pvc-usage          12d
management-storage-policy-single-node-pvc-usage      12d
management-storage-policy-stretched-lite-pvc-usage   12d
management-storage-policy-thin-pvc-usage             12d
vm-encryption-policy-pvc-usage                       12d
vsan-default-storage-policy-pvc-usage                12d
wcpglobal-storage-profile-pvc-usage                  12d
wcplocal-storage-profile-pvc-usage                   11s         <<< storagepolicyusage for policy "wcplocal-storage-profile" created from reconciler

Special notes for your reviewer:

Release note:

Add reconciler for storagepolicyquota CRs in syncer

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: akankshapanse Once this PR has been reviewed and has the lgtm label, please assign divyenpatel for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/kubernetes-sigs/vsphere-csi-driver/blob/master/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.

[deepakkinni]

please rebase this