can't uninstall common services when uninstall meta-operator

[yahliu]

Install meta-operator on OCP4.2 then uninstall it.

[root@knolls-inf common-svcs-sert-tests]# oc get sub meta-operator-app -n yhliu
NAME                PACKAGE             SOURCE                CHANNEL
meta-operator-app   meta-operator-app   opencloud-operators   alpha
[root@knolls-inf common-svcs-sert-tests]# oc delete sub meta-operator-app -n yhliu
subscription.operators.coreos.com "meta-operator-app" deleted
[root@knolls-inf common-svcs-sert-tests]# oc delete clusterserviceversion meta-operator.v0.0.1 -n yhliu
clusterserviceversion.operators.coreos.com "meta-operator.v0.0.1" deleted
[root@knolls-inf common-svcs-sert-tests]# oc get pods -n yhliu
NAME                             READY   STATUS        RESTARTS   AGE
meta-operator-756c747948-b6wcl   0/1     Terminating   0          28h
[root@knolls-inf common-svcs-sert-tests]# oc get pods -n yhliu
No resources found.

[root@knolls-inf common-svcs-sert-tests]# oc get pods -n ibm-cert-manager-operator
NAME                                         READY   STATUS    RESTARTS   AGE
ibm-cert-manager-operator-66f9545ff9-zswgm   1/1     Running   0          26h
[root@knolls-inf common-svcs-sert-tests]# oc get pods -n ibm-licensing-operator
NAME                                      READY   STATUS    RESTARTS   AGE
ibm-licensing-operator-7969d99754-cv54q   1/1     Running   0          18h
[root@knolls-inf common-svcs-sert-tests]#

[morvencao]

/kind bug

[morvencao]

/area config

[horis233]

When uninstalling the meta-operator, we need to remove the metaoperatorset instance first. Then delete the meta-operator. Otherwise, if you delete the meta-operator first, there would be no controller to manage the uninstall logic.

(I am not if there is a way to add a finalizer to the operator deployment to wait for all the cr terminating, I opened an issue in the community)

There is one optional way to clean up all the common service operators if all of them are deployed in the same namespace. We can set metaoperatorset instance as the owner of subscription and CSV of common service operators. Then common service operators can be removed when deleting the meta-operator.

[horis233]

@yahliu

Based on the response from the community and the discussion with @DanielXLee, common services should not be removed when you uninstall csv and subscription directly. When we delete an operator from Openshift operatorhub, it shows Your application will keep running.

If users want to uninstall Common services, they need to do the following steps:

1. Delete the metaoperatorset CR.
2. Delete the Meta operator.

From my understanding, An Operator is like an installer. Users can use it to install an application. If users want to delete the application, they need to use the installer to delete it instead of deleting the installer. Operators are also like this.

FYI @gyliu513 @chenzhiwei @DanielXLee

[yahliu]

@horis233 If I want to clear all services operator and meta-operator , should do below:

1. delete the metaoperatorset cr
2. delete meta-operator subscription and csv. Please correct me if I'm wrong. If the steps are right, I will try it.

[yahliu]

Tried above two steps, can be uninstalled all services after delete the metaoperator cr, but the crds of services will be left.

[root@fixity-inf ~]# oc get crd | grep ibm
authentications.operator.ibm.com                            2020-02-25T06:06:08Z
certmanagers.operator.ibm.com                               2020-02-25T06:05:55Z
clusterservicestatuses.clusterhealth.ibm.com                2020-02-25T06:06:07Z
healthservices.operator.ibm.com                             2020-02-25T06:06:07Z
metaoperatorcatalogs.operator.ibm.com                       2020-02-25T07:07:44Z
metaoperatorconfigs.operator.ibm.com                        2020-02-25T07:07:44Z
metaoperatorsets.operator.ibm.com                           2020-02-25T07:08:50Z
mongodbs.operator.ibm.com                                   2020-02-25T06:06:04Z
oidcclientwatchers.operator.ibm.com                         2020-02-25T06:06:08Z
paps.operator.ibm.com                                       2020-02-25T06:06:08Z
policycontrollers.operator.ibm.com                          2020-02-25T06:06:08Z
policydecisions.operator.ibm.com                            2020-02-25T06:06:08Z
secretwatchers.operator.ibm.com                             2020-02-25T06:06:08Z
securityonboardings.operator.ibm.com                        2020-02-25T06:06:08Z
[root@fixity-inf ~]#

[gyliu513]

@yahliu can you delete the operatorgroup as well?

@chenzhiwei @DanielXLee @horis233 what is the best practise to delete an operator?

[yahliu]

crds still exist after delete operatorgroup. When delete the operator, only need to delete subscription and csv following OCP doc. crds exist will not impact anything, but I'm not sure whether new crds be applied when install again and crds change.

[chenzhiwei]

The CRDs left is OLM's native behavior and meta-operator/odlm has no chance to clean up the CRD resources.

And since we will make ODLM a very general operator to manage operators, so we also can't let ODLM remove tthe CRD resources because customers may have their own managed operator installation.

[gyliu513]

I post a question at here https://github.com/operator-framework/operator-lifecycle-manager/issues/1326

[gyliu513]

There are a lot of discussion at operator-framework/operator-lifecycle-manager#1326 , but I think we cannot enable OLM to delete CRDs in Q1 now, so I think we need to document this in our document: CRDs will not be deleted, and the cluster admin need to delete the CRDs if they do not want to use.

[DanielXLee]

This is common services uninstall doc: https://github.com/IBM/ibm-common-service-operator/blob/master/docs/uninstall.md