red-hat-storage / ocs-operator

Operator for RHOCS
Apache License 2.0
85 stars 184 forks source link

Reducing the storage.k8s.io group privileges #2760

Closed OdedViner closed 3 months ago

OdedViner commented 3 months ago
Procedure:
1. Deploy OCP4.17 [4.17.0-0.nightly-2024-08-19-165854]

2. Deploy ODF4.17 [odf-operator.v4.17.0-80.stable]

3.Verify storagecluster in Ready State
$ oc get storagecluster 
NAME                 AGE     PHASE   EXTERNAL   CREATED AT             VERSION
ocs-storagecluster   5m22s   Ready              2024-08-22T15:07:34Z   4.17.0

4.Check clusterrole status:
+kubebuilder:rbac:groups=storage.k8s.io,resources=storageclasses,verbs=*
$ oc get clusterrole ocs-operator.v4.17.0-80.-5ZFqOum5B7idpvPbl6Z3TfkwxJZUIirvkN8UEe -o yaml | grep storageclasses -C 10

- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  verbs:
  - '*'
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch

5. Add create and  delete verbs to ocs-operator clusterrole:
$ oc edit clusterrole ocs-operator.v4.17.0-80.-5ZFqOum5B7idpvPbl6Z3TfkwxJZUIirvkN8UEe
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  verbs:
  - create
  - delete

6.Delete ocs-operator pod and check ocs-operator pod logs:
$ oc delete pod ocs-operator-6fbf456698-kn4r7 
pod "ocs-operator-6fbf456698-kn4r7" deleted

$ oc logs ocs-operator-6fbf456698-ldfhh
{"level":"error","ts":"2024-08-22T15:25:30Z","logger":"controllers.StorageCluster","msg":"unable to update clusterroles for metrics exporter","Request.Namespace":"openshift-storage","Request.Name":"ocs-storagecluster","error":"clusterroles.rbac.authorization.k8s.io \"ocs-metrics-exporter\" is forbidden: user \"system:serviceaccount:openshift-storage:ocs-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:openshift-storage\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"storage.k8s.io\"], Resources:[\"storageclasses\"], Verbs:[\"watch\"]}","stacktrace":"github.com/red-hat-storage/ocs-operator/v4/controllers/storagecluster.(*StorageClusterReconciler).enableMetricsExporter\n\t/remote-source/app/controllers/storagecluster/exporter.go:56\ngithub.com/red-hat-storage/ocs-operator/v4/controllers/storagecluster.(*StorageClusterReconciler).reconcilePhases\n\t/remote-source/app/controllers/storagecluster/reconcile.go:598\ngithub.com/red-hat-storage/ocs-operator/v4/controllers/storagecluster.(*StorageClusterReconciler).Reconcile\n\t/remote-source/app/controllers/storagecluster/reconcile.go:178\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222"}

7.Add "watch" verb to ocs-operator clusterrole:
$ oc edit clusterrole ocs-operator.v4.17.0-80.-5ZFqOum5B7idpvPbl6Z3TfkwxJZUIirvkN8UEe
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  verbs:
  - create
  - delete
  - watch

8.Delete ocs-operator pod and check ocs-operator pod logs:
oviner~$ oc delete pod ocs-operator-6fbf456698-ldfhh 
pod "ocs-operator-6fbf456698-ldfhh" deleted

No Error in logs ["oc logs ocs-operator"]

9.Running Stoageclass test [ storage class creation,pvc and pod with rbd/cephfs]
https://github.com/red-hat-storage/ocs-ci/blob/master/tests/functional/storageclass/test_create_storage_class_pvc.py
OdedViner commented 3 months ago

@OdedViner Can you pls fix the failures?

Fixed gen-latest-csv ci/prow/ocs-operator-bundle-e2e-aws — Job failed

iamniting commented 3 months ago

/test ocs-operator-bundle-e2e-aws

malayparida2000 commented 3 months ago

/retest

iamniting commented 3 months ago

/test ocs-operator-bundle-e2e-aws

openshift-ci[bot] commented 3 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: iamniting, OdedViner

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/red-hat-storage/ocs-operator/blob/main/OWNERS)~~ [iamniting] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
malayparida2000 commented 3 months ago

/retest