Reducing the quota.openshift.io group privileges

[OdedViner]

Procedure:
1. Deploy OCP4.17 [4.17.0-0.nightly-2024-08-19-165854]

2. Deploy ODF4.17 [4.17.0-84.stable]

3.Verify storagecluster in Ready State
$ oc get storagecluster
NAME                 AGE    PHASE   EXTERNAL   CREATED AT             VERSION
ocs-storagecluster   4d3h   Ready              2024-08-26T08:39:13Z   4.17.0

4.Check clusterrole status:
// +kubebuilder:rbac:groups=quota.openshift.io,resources=clusterresourcequotas,verbs=*

5. Add "create" and "delete" verbs
$ oc edit clusterrole ocs-operator.v4.17.0-84.-8Luvew6SyBvGMmyBWPm4T89mZWV6JigkKo20ha
- apiGroups:
  - quota.openshift.io
  resources:
  - clusterresourcequotas
  verbs:
  - create
  - delete

6.Delete ocs-operator pod and check ocs-operator pod logs:
$ oc delete pod ocs-operator

$ oc logs ocs-operator
{"level":"error","ts":"2024-08-30T12:26:03Z","logger":"controllers.StorageCluster","msg":"unable to update clusterroles for metrics exporter","Request.Namespace":"openshift-storage","Request.Name":"ocs-storagecluster","error":"clusterroles.rbac.authorization.k8s.io \"ocs-metrics-exporter\" is forbidden: user \"system:serviceaccount:openshift-storage:ocs-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:openshift-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"quota.openshift.io\"], Resources:[\"clusterresourcequotas\"], Verbs:[\"get\" \"list\" \"watch\"]}","stacktrace":"github.com/red-hat-storage/ocs-operator/v4/controllers/storagecluster.(*StorageClusterReconciler).enableMetricsExporter\n\t/remote-source/app/controllers/storagecluster/exporter.go:56\ngithub.com/red-hat-storage/ocs-operator/v4/controllers/storagecluster.(*StorageClusterReconciler).reconcilePhases\n\t/remote-source/app/controllers/storagecluster/reconcile.go:598\ngithub.com/red-hat-storage/ocs-operator/v4/controllers/storagecluster.(*StorageClusterReconciler).Reconcile\n\t/remote-source/app/controllers/storagecluster/reconcile.go:178\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222"}

7.Add "watch" and "list" and "watch"  verbs to ocs-operator clusterrole:
$ oc edit clusterrole ocs-operator.v4.17.0-84.-8Luvew6SyBvGMmyBWPm4T89mZWV6JigkKo20ha
- apiGroups:
  - quota.openshift.io
  resources:
  - clusterresourcequotas
  verbs:
  - create
  - delete
  - get
  - list
  - watch

8.Delete ocs-operator pod and check ocs-operator pod logs:
$ oc delete pod ocs-operator

9. Running OCS-CI test:
tests/functional/pv/pv_services/test_overprovision_level_policy_control.py

{"level":"info","ts":"2024-08-30T12:33:15Z","logger":"controllers.StorageCluster","msg":"creating ClusterResourceQuota ocs-storagecluster-ceph-rbd-quota-sc-test with map[ocs-storagecluster-ceph-rbd.storageclass.storage.k8s.io/requests.storage:{i:{value:8589934592 scale:0} d:{Dec:<nil>} s: Format:BinarySI}]","Request.Namespace":"openshift-storage","Request.Name":"ocs-storagecluster"}

10. Change code:
// +kubebuilder:rbac:groups=quota.openshift.io,resources=clusterresourcequotas,verbs=create;delete;list;watch;get

11.make gen-latest-csv:
export REGISTRY_NAMESPACE=ocs-dev
export IMAGE_TAG=latest
make gen-latest-csv

[OdedViner]

We also need an update

https://github.com/red-hat-storage/ocs-operator/blob/main/controllers/storagecluster/storagequota.go#L54

@iamniting Done

[openshift-ci[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: iamniting, OdedViner

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/red-hat-storage/ocs-operator/blob/main/OWNERS)~~ [iamniting] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment