red-hat-storage / ocs-operator

Operator for RHOCS

Apache License 2.0

86 stars 184 forks source link

Available CRDs check feature (with script) #2741

Closed raaizik closed 1 month ago

raaizik commented 1 month ago

Changes

Reasons for this enhancement:

A controller cannot set up a watch for a CRD that is not installed on the cluster, trying to set up a watch will panic the operator
There is no known way, that we are aware of, to add a watch later without client cache issue

How does the enhancement work around the issue:

On start up, detect which CRD are avail (out of a fixed list) and skip watches for ones that are not avail
At the start each reconcile iteration, revalidate which CRD are now available. If a CRD of interest is now avail, exit the op with a known exit code (42)
Have the pod command detect the exit code and if it is the known exist code (42), restart the process

This process will guarantee that the pod does restart when a new CRD of interest becomes available. This in turn helps to avoid the following issue:

Pod will not get into CrushLoopBackoff state
There will be no change that the pod becomes unschedulable after restart as of missing resources

Note

2712 is considered as a potential enhancement by @iamniting

openshift-ci[bot] commented 1 month ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: raaizik Once this PR has been reviewed and has the lgtm label, please assign blaineexe for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/red-hat-storage/ocs-operator/blob/main/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

raaizik commented 1 month ago

/cc @nb-ohad @umangachapagain

openshift-ci[bot] commented 1 month ago

@raaizik: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/ocs-operator-bundle-e2e-aws	ba19b8b6b4c09bf96781230235e7b304779fb4cd	link	true	`/test ocs-operator-bundle-e2e-aws`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).