Ansible operator does not detect unauthorized changes to resources that are managed by the operator on multi namespace mode

Bug Report

What did you do?

I am upgrading an ansible operator from old v0.18.1to latest v1.4.2 (adding also multi namespace support at the CSV).

I deploy the ansible operator with OLM with MultiNamespace support (setting 2 namespaces on targetNamespaces field at OperatorGroup resource), I create a CR on 2 diferent namespaces, and operator reconciles them by creating all needed dependant objects on every namespace

However, if I delete manually any object belonging to that created CRs on any on the namespaces, ansible operator doesn't detect they are missing, and never run a reconcile to recreate them.

If I run the operator locally watching both namespaces (make run WATCH_NAMESPACE=ns1,ns2 target), that way the ansible operator works fine, so any manual object deletion triggers immediately a reconcile loop that recreates the missing objects.

It has the same behaviour with operator-sdk v1.5.0

What did you expect to see?

After a manual deletion of a dependant object from a CR, I expect the operator to recreate it immediately.

What did you see instead? Under which circumstances?

Nothing, deleted objects that belongs to a CR are never recreated, unless I update the CRs, so that way ansible runs the reconcile.

Environment

Operator type:

/language ansible

Kubernetes cluster type: Openshift v4.7.0

$ operator-sdk version

operator-sdk version: "v1.4.2", commit: "4b083393be65589358b3e0416573df04f4ae8d9b", kubernetes version: "1.19.4", go version: "go1.15.5", GOOS: "linux", GOARCH: "amd64"

And:

operator-sdk version: "v1.5.0", commit: "98f30d59ade2d911a7a8c76f0169a7de0dec37a0", kubernetes version: "1.19.4", go version: "go1.15.5", GOOS: "linux", GOARCH: "amd64"

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.13-dispatcher", GitCommit:"fd22db44e150011eccc8729db223945384460143", GitTreeState:"clean", BuildDate:"2020-07-24T07:27:52Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0+bd9e442", GitCommit:"bd9e4421804c212e6ac7ee074050096f08dda543", GitTreeState:"clean", BuildDate:"2021-02-11T23:05:38Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

Possible Solution

Additional context

@slopezz would you be able to paste the CSV and operator deployment that olm creates? Looks like it's probably just not setting that environment up properly but curious to see what it's actually doing

Sure @fabianvf , it is possible that I'm doing something wrong (first time running the operator with multi namespace support), I am using the following CSV https://github.com/3scale/prometheus-exporter-operator/blob/upgrade-operator-sdk-1.4/bundle/manifests/prometheus-exporter-operator.clusterserviceversion.yaml

I converted the default ClusterRole/ClusterRoleBinding to Role/RoleBinding (don't want to force someone to use ClusterRole with access to every namespace on a cluster)
Added true to the MultiNamespace install type support

And I have installed by creating 2 namespaces:

oc create namespace test-prom
oc create namespace test-prom2

And creating the following CatalogSource, OperatorGroup and Subscription:

---
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: prometheus-exporter-operator-catalog
namespace: openshift-marketplace
spec:
sourceType: grpc
image: quay.io/3scale/prometheus-exporter-operator-bundle:catalog
displayName: Prometheus Exporter Operator
updateStrategy:
registryPoll:
  interval: 30m
---
apiVersion: operators.coreos.com/v1alpha2
kind: OperatorGroup
metadata:
name: prometheus-exporter
namespace: test-prom
spec:
targetNamespaces:
- test-prom
- test-prom2
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: prometheus-exporter-operator
namespace: test-prom
spec:
channel: alpha
installPlanApproval: Automatic
name: prometheus-exporter-operator
source: prometheus-exporter-operator-catalog
sourceNamespace: openshift-marketplace
config:
env:
- name: ANSIBLE_DEBUG_LOGS
  value: "true"

And finally created 2 custom resources (one on every namespace):

apiVersion: monitoring.3scale.net/v1alpha1
kind: PrometheusExporter
metadata:
name: example-memcached
namespace: test-prom
spec:
type: memcached
dbHost: your-memcached-host
dbPort: 11211
grafanaDashboard:
label:
  key: monitoring-key
  value: middleware
----
apiVersion: monitoring.3scale.net/v1alpha1
kind: PrometheusExporter
metadata:
name: example-memcached2
namespace: test-prom2
spec:
type: memcached
dbHost: your-memcached
dbPort: 11211
grafanaDashboard:
label:
  key: monitoring-key
  value: middleware

The result is that every CR from 2 namespace is correctly reconciled by the same operator on just one namespace, but then, if you delete any dependant object on any namespace, operator don't get noticed and never run a reconcile (reconcile only runs only if I update the CRs).

@slopezz could you also paste the actual operator Deployment (the one defined as prometheus-exporter-operator-controller-manager in the CSV), I want to see what the metadata.annotations['olm.targetNamespaces'] that is being used to set the WATCH_NAMESPACE envvar looks like. Also might be useful to see what the roles/role bindings in the target namespace look like, to see if those are being configured as expected by OLM

@fabianvf I copy paste the whole CSV:

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  annotations:
    alm-examples: |-
      [
        {
          "apiVersion": "monitoring.3scale.net/v1alpha1",
          "kind": "PrometheusExporter",
          "metadata": {
            "name": "example-memcached"
          },
          "spec": {
            "dbHost": "your-memcached-host",
            "dbPort": 11211,
            "grafanaDashboard": {
              "label": {
                "key": "autodiscovery",
                "value": "enabled"
              }
            },
            "type": "memcached"
          }
        }
      ]
    capabilities: Deep Insights
    categories: Monitoring
    certified: "false"
    containerImage: quay.io/3scale/prometheus-exporter-operator
    createdAt: "2020-06-08 00:00:00"
    description: Operator to setup 3rd party prometheus exporters, with a collection of grafana dashboards
    operators.operatorframework.io/builder: operator-sdk-v1.4.2
    operators.operatorframework.io/project_layout: ansible.sdk.operatorframework.io/v1
    repository: https://github.com/3scale/prometheus-exporter-operator
    support: Red Hat, Inc.
  name: prometheus-exporter-operator.v0.3.0-alpha.2
  namespace: placeholder
spec:
  apiservicedefinitions: {}
  customresourcedefinitions:
    owned:
    - description: Configures a prometheus exporter to monitor a memcached instance
      displayName: PrometheusExporter
      kind: PrometheusExporter
      name: prometheusexporters.monitoring.3scale.net
      version: v1alpha1
  description: |
    A Kubernetes Operator based on the Operator SDK to centralize the setup of 3rd party prometheus exporters on **Kubernetes/OpenShift**, with a collection of grafana dashboards.
    You can setup different prometheus exporters to monitor the **internals from different databases**, **HTTP/TCP endpoints** (availability, latency, SSL/TLS certificate expiration...), or even any available **cloudwatch metric from any AWS Service**, by just providing a few parameters like **dbHost** or **dbPort** (operator manages the container image, port, argument, command, volumes... and also prometheus **ServiceMonitor** and **GrafanaDashboard** k8s objects).
    Current prometheus exporters types supported, managed by same prometheus-exporter-operator:
    * memcached
    * redis
    * mysql
    * postgresql
    * sphinx
    * es (elasticsearch)
    * cloudwatch
    * probe (blackbox)
    The operator manages the lifecycle of the following objects:
    * Deployment (one per CR)
    * Service (one per CR)
    * ServiceMonitor (optional, one per CR)
    * GrafanaDashboard (optional, one per Namespace)
    ### Documentation
    Documentation can be found on our [website](https://github.com/3scale/prometheus-exporter-operator#documentation).
    ### Getting help
    If you encounter any issues while using operator, you can create an issue on our [website](https://github.com/3scale/prometheus-exporter-operator) for bugs, enhancements, or other requests.
    ### Contributing
    You can contribute by:
    * Raising any issues you find using Prometheus Exporter Operator
    * Fixing issues by opening [Pull Requests](https://github.com/3scale/prometheus-exporter-operator/pulls)
    * Submitting a patch or opening a PR
    * Improving [documentation](https://github.com/3scale/prometheus-exporter-operator)
    * Talking about Prometheus Exporter Operator
    All bugs, tasks or enhancements are tracked as [GitHub issues](https://github.com/3scale/prometheus-exporter-operator/issues).
    ### License
    Prometheus Exporter Operator is licensed under the [Apache 2.0 license](https://github.com/3scale/prometheus-exporter-operator/blob/master/LICENSE)
  displayName: Prometheus Exporter Operator
  icon:
  - base64data: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxvbmU9Im5vIj8+CjxzdmcKICAgeG1sbnM6ZGM9Imh0dHA6Ly9wdXJsLm9yZy9kYy9lbGVtZW50cy8xLjEvIgogICB4bWxuczpjYz0iaHR0cDovL2NyZWF0aXZlY29tbW9ucy5vcmcvbnMjIgogICB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiCiAgIHhtbG5zOnN2Zz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciCiAgIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIKICAgeG1sbnM6c29kaXBvZGk9Imh0dHA6Ly9zb2RpcG9kaS5zb3VyY2Vmb3JnZS5uZXQvRFREL3NvZGlwb2RpLTAuZHRkIgogICB4bWxuczppbmtzY2FwZT0iaHR0cDovL3d3dy5pbmtzY2FwZS5vcmcvbmFtZXNwYWNlcy9pbmtzY2FwZSIKICAgaW5rc2NhcGU6dmVyc2lvbj0iMC45Mi40ICh1bmtub3duKSIKICAgaW5rc2NhcGU6ZXhwb3J0LXlkcGk9IjQxMC4xOSIKICAgaW5rc2NhcGU6ZXhwb3J0LXhkcGk9IjQxMC4xOSIKICAgaW5rc2NhcGU6ZXhwb3J0LWZpbGVuYW1lPSIvVXNlcnMvcmFlbC9Eb3dubG9hZHMvcHJvbWV0aGV1cy1leHBvcnRlci5wbmciCiAgIHNvZGlwb2RpOmRvY25hbWU9InByb21ldGhldXMtZXhwb3J0ZXItM3NjYWxlLW5ldy5zdmciCiAgIGlkPSJzdmc4NDMiCiAgIHByZXNlcnZlQXNwZWN0UmF0aW89InhNaWRZTWlkIG1lZXQiCiAgIHZpZXdCb3g9IjAgMCA3MzYuNDI1NzggNjE5Ljc0ODQ3IgogICBoZWlnaHQ9IjYxOS43NDg0N3B0IgogICB3aWR0aD0iNzM2LjQyNTc4cHQiCiAgIHZlcnNpb249IjEuMCI+CiAgPG1ldGFkYXRhCiAgICAgaWQ9Im1ldGFkYXRhODQ5Ij4KICAgIDxyZGY6UkRGPgogICAgICA8Y2M6V29yawogICAgICAgICByZGY6YWJvdXQ9IiI+CiAgICAgICAgPGRjOmZvcm1hdD5pbWFnZS9zdmcreG1sPC9kYzpmb3JtYXQ+CiAgICAgICAgPGRjOnR5cGUKICAgICAgICAgICByZGY6cmVzb3VyY2U9Imh0dHA6Ly9wdXJsLm9yZy9kYy9kY21pdHlwZS9TdGlsbEltYWdlIiAvPgogICAgICA8L2NjOldvcms+CiAgICA8L3JkZjpSREY+CiAgPC9tZXRhZGF0YT4KICA8ZGVmcwogICAgIGlkPSJkZWZzODQ3IiAvPgogIDxzb2RpcG9kaTpuYW1lZHZpZXcKICAgICBpbmtzY2FwZTpjdXJyZW50LWxheWVyPSJzdmc4NDMiCiAgICAgaW5rc2NhcGU6d2luZG93LW1heGltaXplZD0iMSIKICAgICBpbmtzY2FwZTp3aW5kb3cteT0iMjciCiAgICAgaW5rc2NhcGU6d2luZG93LXg9IjAiCiAgICAgaW5rc2NhcGU6Y3k9IjM1Ny4wNDcxNyIKICAgICBpbmtzY2FwZTpjeD0iMzc0LjM3OTciCiAgICAgaW5rc2NhcGU6em9vbT0iMC42ODc3ODIyNCIKICAgICBpbmtzY2FwZTpsb2NrZ3VpZGVzPSJ0cnVlIgogICAgIHNob3dncmlkPSJmYWxzZSIKICAgICBpZD0ibmFtZWR2aWV3ODQ1IgogICAgIGlua3NjYXBlOndpbmRvdy1oZWlnaHQ9IjEzNzYiCiAgICAgaW5rc2NhcGU6d2luZG93LXdpZHRoPSIyNTYwIgogICAgIGlua3NjYXBlOnBhZ2VzaGFkb3c9IjIiCiAgICAgaW5rc2NhcGU6cGFnZW9wYWNpdHk9IjAiCiAgICAgZ3VpZGV0b2xlcmFuY2U9IjEwIgogICAgIGdyaWR0b2xlcmFuY2U9IjEwIgogICAgIG9iamVjdHRvbGVyYW5jZT0iMTAiCiAgICAgYm9yZGVyb3BhY2l0eT0iMSIKICAgICBpbmtzY2FwZTpkb2N1bWVudC1yb3RhdGlvbj0iMCIKICAgICBib3JkZXJjb2xvcj0iIzY2NjY2NiIKICAgICBwYWdlY29sb3I9IiNmZmZmZmYiIC8+CiAgPHBhdGgKICAgICBkPSJtIDcxNS4xMDM4NiwzMS43NDg0NzYgYyAtMTEuNSw0LjkgLTI3LjMsMTEuNSAtMzUsMTQuNyAtMjAuNiw4LjcgLTUwLDIxLjEgLTY2LjUsMjguMSAtMTcuNSw3LjUgLTM4LjEsMTYuMiAtNzEuMywzMC4wMDAwMDQgLTEzLjYsNS43IC0yNC42LDEwLjcgLTI0LjUsMTEuMSAwLjIsMC4zIDEwLjcsOS40IDIzLjMsMjAuMSAxMi43LDEwLjggMjQuOSwyMS4zIDI3LjIsMjMuNCBsIDQuMywzLjkgLTYuNyw3LjUgYyAtMzEuOSwzNi4xIC04MS41LDg2LjkgLTExMi44LDExNS43IC0xMTAuNiwxMDEuNiAtMjIyLjQsMTcwLjIgLTMyNywyMDAuOCAtMTQuNyw0LjMgLTQxLjEwMDAwMSwxMC42IC00OS41MDAwMDEsMTEuOCAtMS45LDAuMyAtNi40LDEgLTEwLDEuNSAtMy42LDAuNiAtOS40LDEuNCAtMTMsMS43IC0zLjYsMC40IC05LjQsMSAtMTMsMS41IC0zLjYsMC40IC0xNC4yLDAuNyAtMjMuNywwLjYgLTkuNTAwMDAwMywtMC4xIC0xNy4xMDAwMDAzMiwwIC0xNi45MDAwMDAzMiwwLjEgMS40MDAwMDAwMiwxLjQgMzguMzAwMDAwMzIsNi45IDU2LjAwMDAwMDMyLDguMyAxMS41LDEgNTMuNTAwMDAxLDEuMyA2NC4xMDAwMDEsMC41IDIuOCwtMC4yIDkuMywtMC42IDE0LjUsLTEgNS4yLC0wLjMgMTAuNiwtMC44IDEyLC0xIDEuNCwtMC4yIDUuMiwtMC43IDguNSwtMSAzNy40LC0zLjkgODUuNiwtMTQuOSAxMjYuNSwtMjkuMSA5MS40LC0zMS42IDE3Ny4xLC04Mi45IDI2MSwtMTU2LjUgMjAuNSwtMTcuOSA2OC4xLC02NS4yIDg2LjQsLTg1LjcgbCAxMi44LC0xNC4yIDEuOCwyLjIgYyAwLjksMS4zIDEwLjgsMTQgMjEuOSwyOC4zIDExLjEsMTQuMyAyMC41LDI2LjEgMjEsMjYuMyAwLjQsMC4xIDEuNSwtMy43IDIuNSwtOC41IDAuOSwtNC45IDYuNCwtMzMuMSAxMi4xLC02Mi44IDUuNywtMjkuNyAxMi45LC02Ny4xIDE2LC04MyAxNywtODcuODAwMDA0IDE5LjYsLTEwMS43MDAwMDQgMTkuMywtMTAyLjkwMDAwNCAtMC4yLC0wLjggLTgsMiAtMjEuMyw3LjYgeiIKICAgICBpZD0icGF0aDgzNSIKICAgICBpbmtzY2FwZTpjb25uZWN0b3ItY3VydmF0dXJlPSIwIgogICAgIHN0eWxlPSJmaWxsOiM1YTVhNWE7ZmlsbC1vcGFjaXR5OjE7c3Ryb2tlOm5vbmU7c3Ryb2tlLXdpZHRoOjAuMSIgLz4KICA8cGF0aAogICAgIGQ9Im0gMjIuMjAzODU5LDE1Ni41NDg0OCBjIDIsMTUgMTkuNiw2OS42IDI1LDc3LjUgMC44LDEuMSAxLjQsMi41IDEuNCwzLjMgMCwwLjcgMiw1LjQgNC41LDEwLjQgMi41LDUgNC41LDkuNSA0LjUsOS45IDAsMS4xIDE1LjUsMjkuNCAyMyw0MS45IDkuMywxNS43IDM2LjEwMDAwMSw1NC44IDM4LjkwMDAwMSw1NyAwLjQsMC4zIDEuOCwyLjEgMy4xLDQgMS4zLDEuOSAyLjcsMy43IDMsNCAwLjMsMC4zIDIuNCwyLjcgNC41LDUuNSAyLjIsMi43IDQuMiw1LjIgNC41LDUuNSAwLjMsMC4zIDMuNCwzLjkgNyw4IDExLjgsMTMuNyAxNy4yLDE5LjMgMzguOCw0MC43IDExLjgsMTEuNyAyMi4xLDIxLjMgMjIuNywyMS4zIDEuNywwIDYzLjksLTM0LjcgNjQuMywtMzUuOCAwLjEsLTAuNSAtNC44LC00IC0xMSwtNy43IC04MS42LC00OS4zIC0xNTYuNjAwMDAxLC0xMTcgLTE5OC41MDAwMDEsLTE3OS41IC0xNS4zLC0yMi44IC0yNywtNDUuMyAtMzUuMiwtNjcuNSAtMSwtMi43IC0xLC0yLjYgLTAuNSwxLjUgeiIKICAgICBpZD0icGF0aDgzNyIKICAgICBpbmtzY2FwZTpjb25uZWN0b3ItY3VydmF0dXJlPSIwIgogICAgIHN0eWxlPSJmaWxsOiNlYzdhMDg7ZmlsbC1vcGFjaXR5OjE7c3Ryb2tlOm5vbmU7c3Ryb2tlLXdpZHRoOjAuMSIgLz4KICA8cGF0aAogICAgIGQ9Im0gMzE4LjMwMzg2LDQ3OS40NDg0OCBjIC0yNi44LDkuNyAtNDguNywxNy45IC00OC43LDE4LjMgMCwxLjIgMzEuNSwyMS42IDUyLjgsMzQuMiA1NCwzMiAxMjkuOSw2NiAxOTAuMiw4NS4xIGwgOC41LDIuNyAyNywtMC42IGMgMjUuOSwtMC42IDI3LjgsLTAuOCA0OCwtNC42IDQwLjUsLTcuNiA2OC43LC0xNy45IDg5LjIsLTMyLjYgNi4xLC00LjMgMTAuOSwtOC4xIDEwLjcsLTguNCAtMS40LC0xLjQgLTUyLjgsLTE2IC04OS40LC0yNS40IC0xNCwtMy43IC0yNi40LC03IC0yNy41LC03LjUgLTEuMSwtMC41IC02LjEsLTEuOCAtMTEuMSwtMyAtMTAsLTIuMyAtNjQuNCwtMjAuMyAtOTAuNywtMzAgLTMwLjgsLTExLjQgLTcyLjIsLTI4LjUgLTEwNC43LC00My40IGwgLTUuNiwtMi41IHoiCiAgICAgaWQ9InBhdGg4MzkiCiAgICAgaW5rc2NhcGU6Y29ubmVjdG9yLWN1cnZhdHVyZT0iMCIKICAgICBzdHlsZT0iZmlsbDojZWM3YTA4O2ZpbGwtb3BhY2l0eToxO3N0cm9rZTpub25lO3N0cm9rZS13aWR0aDowLjEiIC8+Cjwvc3ZnPgo=
    mediatype: image/svg+xml
  install:
    spec:
      clusterPermissions:
      - rules:
        - apiGroups:
          - authentication.k8s.io
          resources:
          - tokenreviews
          verbs:
          - create
        - apiGroups:
          - authorization.k8s.io
          resources:
          - subjectaccessreviews
          verbs:
          - create
        serviceAccountName: prometheus-exporter-operator-controller-manager
      deployments:
      - name: prometheus-exporter-operator-controller-manager
        spec:
          replicas: 1
          selector:
            matchLabels:
              control-plane: controller-manager
          strategy: {}
          template:
            metadata:
              labels:
                control-plane: controller-manager
            spec:
              containers:
              - args:
                - --secure-listen-address=0.0.0.0:8443
                - --upstream=http://127.0.0.1:8080/
                - --logtostderr=true
                - --v=10
                image: gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0
                name: kube-rbac-proxy
                ports:
                - containerPort: 8443
                  name: https
                resources: {}
              - args:
                - --metrics-addr=127.0.0.1:8080
                - --enable-leader-election
                - --leader-election-id=prometheus-exporter-operator
                env:
                - name: ANSIBLE_GATHERING
                  value: explicit
                - name: WATCH_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.annotations['olm.targetNamespaces']
                image: quay.io/3scale/prometheus-exporter-operator:v0.3.0-alpha.2
                livenessProbe:
                  httpGet:
                    path: /readyz
                    port: 6789
                  initialDelaySeconds: 15
                  periodSeconds: 20
                  timeoutSeconds: 5
                name: manager
                readinessProbe:
                  httpGet:
                    path: /healthz
                    port: 6789
                  initialDelaySeconds: 5
                  periodSeconds: 10
                  timeoutSeconds: 5
                resources: {}
              serviceAccountName: prometheus-exporter-operator-controller-manager
              terminationGracePeriodSeconds: 10
      permissions:
      - rules:
        - apiGroups:
          - ""
          resources:
          - configmaps
          verbs:
          - get
          - list
          - watch
          - create
          - update
          - patch
          - delete
        - apiGroups:
          - ""
          resources:
          - events
          verbs:
          - create
          - patch
        - apiGroups:
          - ""
          resources:
          - secrets
          - pods
          - pods/exec
          - pods/log
          - pods
          - services
          - services/finalizers
          - endpoints
          - persistentvolumeclaims
          - events
          - configmaps
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - apps
          resources:
          - deployments
          - daemonsets
          - replicasets
          - statefulsets
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - ""
          resources:
          - services
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - ""
          resources:
          - secrets
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - ""
          resources:
          - configmaps
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - monitoring.coreos.com
          resources:
          - servicemonitors
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - integreatly.org
          resources:
          - grafanadashboards
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - monitoring.3scale.net
          resources:
          - prometheusexporters
          - prometheusexporters/status
          - prometheusexporters/finalizers
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        serviceAccountName: prometheus-exporter-operator-controller-manager
    strategy: deployment
  installModes:
  - supported: true
    type: OwnNamespace
  - supported: true
    type: SingleNamespace
  - supported: true
    type: MultiNamespace
  - supported: false
    type: AllNamespaces
  keywords:
  - monitoring
  - metrics
  - observability
  - prometheus
  - exporter
  - grafana
  - mysql
  - postgresql
  - memcached
  - redis
  - sphinx
  - elasticsearch
  - cloudwatch
  - blackbox
  links:
  - name: GitHub
    url: https://github.com/3scale/prometheus-exporter-operator
  maintainers:
  - email: 3scale-operations@redhat.com
    name: 3scale Ops
  maturity: alpha
  provider:
    name: Red Hat
    url: https://www.redhat.com
  version: 0.3.0-alpha.2

On the first test-prom NS:

$ oc get role prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df -oyaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  creationTimestamp: "2021-03-12T17:38:05Z"
  labels:
    olm.owner: prometheus-exporter-operator.v0.3.0-alpha.2
    olm.owner.kind: ClusterServiceVersion
    olm.owner.namespace: test-prom
    operators.coreos.com/prometheus-exporter-operator.test-prom: ""
  managedFields:
  - apiVersion: rbac.authorization.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:olm.owner: {}
          f:olm.owner.kind: {}
          f:olm.owner.namespace: {}
        f:ownerReferences:
          .: {}
          k:{"uid":"b64d73e4-8699-47d5-9f9e-932089100155"}:
            .: {}
            f:apiVersion: {}
            f:blockOwnerDeletion: {}
            f:controller: {}
            f:kind: {}
            f:name: {}
            f:uid: {}
      f:rules: {}
    manager: catalog
    operation: Update
    time: "2021-03-12T17:38:05Z"
  - apiVersion: rbac.authorization.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          f:operators.coreos.com/prometheus-exporter-operator.test-prom: {}
    manager: olm
    operation: Update
    time: "2021-03-12T17:38:06Z"
  name: prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
  namespace: test-prom
  ownerReferences:
  - apiVersion: operators.coreos.com/v1alpha1
    blockOwnerDeletion: false
    controller: false
    kind: ClusterServiceVersion
    name: prometheus-exporter-operator.v0.3.0-alpha.2
    uid: b64d73e4-8699-47d5-9f9e-932089100155
  resourceVersion: "1691638"
  selfLink: /apis/rbac.authorization.k8s.io/v1/namespaces/test-prom/roles/prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
  uid: 99f2508c-570d-4078-81bb-1085a4166d69
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
  - delete
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - ""
  resources:
  - secrets
  - pods
  - pods/exec
  - pods/log
  - pods
  - services
  - services/finalizers
  - endpoints
  - persistentvolumeclaims
  - events
  - configmaps
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - deployments
  - daemonsets
  - replicasets
  - statefulsets
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - monitoring.coreos.com
  resources:
  - servicemonitors
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - integreatly.org
  resources:
  - grafanadashboards
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - monitoring.3scale.net
  resources:
  - prometheusexporters
  - prometheusexporters/status
  - prometheusexporters/finalizers
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch

$ oc get rolebinding prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df -oyaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  creationTimestamp: "2021-03-12T17:38:05Z"
  labels:
    olm.owner: prometheus-exporter-operator.v0.3.0-alpha.2
    olm.owner.kind: ClusterServiceVersion
    olm.owner.namespace: test-prom
    operators.coreos.com/prometheus-exporter-operator.test-prom: ""
  managedFields:
  - apiVersion: rbac.authorization.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:olm.owner: {}
          f:olm.owner.kind: {}
          f:olm.owner.namespace: {}
        f:ownerReferences:
          .: {}
          k:{"uid":"b64d73e4-8699-47d5-9f9e-932089100155"}:
            .: {}
            f:apiVersion: {}
            f:blockOwnerDeletion: {}
            f:controller: {}
            f:kind: {}
            f:name: {}
            f:uid: {}
      f:roleRef:
        f:apiGroup: {}
        f:kind: {}
        f:name: {}
      f:subjects: {}
    manager: catalog
    operation: Update
    time: "2021-03-12T17:38:05Z"
  - apiVersion: rbac.authorization.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          f:operators.coreos.com/prometheus-exporter-operator.test-prom: {}
    manager: olm
    operation: Update
    time: "2021-03-12T17:38:06Z"
  name: prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
  namespace: test-prom
  ownerReferences:
  - apiVersion: operators.coreos.com/v1alpha1
    blockOwnerDeletion: false
    controller: false
    kind: ClusterServiceVersion
    name: prometheus-exporter-operator.v0.3.0-alpha.2
    uid: b64d73e4-8699-47d5-9f9e-932089100155
  resourceVersion: "1691637"
  selfLink: /apis/rbac.authorization.k8s.io/v1/namespaces/test-prom/rolebindings/prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
  uid: b2aca4da-7b38-44b6-b7d1-74d8970b6bcc
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
subjects:
- kind: ServiceAccount
  name: prometheus-exporter-operator-controller-manager
  namespace: test-prom

On the second test-prom2 NS:

$ oc get roles prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df -oyaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  creationTimestamp: "2021-03-12T17:38:06Z"
  labels:
    olm.copiedFrom: test-prom
    olm.owner: prometheus-exporter-operator.v0.3.0-alpha.2
    olm.owner.kind: ClusterServiceVersion
    olm.owner.namespace: test-prom2
  managedFields:
  - apiVersion: rbac.authorization.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:ownerReferences:
          k:{"uid":"b64d73e4-8699-47d5-9f9e-932089100155"}:
            .: {}
            f:apiVersion: {}
            f:blockOwnerDeletion: {}
            f:controller: {}
            f:kind: {}
            f:name: {}
            f:uid: {}
    manager: catalog
    operation: Update
    time: "2021-03-12T17:38:05Z"
  - apiVersion: rbac.authorization.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:olm.copiedFrom: {}
          f:olm.owner: {}
          f:olm.owner.kind: {}
          f:olm.owner.namespace: {}
        f:ownerReferences:
          .: {}
          k:{"uid":"d2980a47-3adf-40d4-a5de-431fd30b620d"}:
            .: {}
            f:apiVersion: {}
            f:blockOwnerDeletion: {}
            f:controller: {}
            f:kind: {}
            f:name: {}
            f:uid: {}
      f:rules: {}
    manager: olm
    operation: Update
    time: "2021-03-12T17:38:06Z"
  name: prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
  namespace: test-prom2
  ownerReferences:
  - apiVersion: operators.coreos.com/v1alpha1
    blockOwnerDeletion: false
    controller: false
    kind: ClusterServiceVersion
    name: prometheus-exporter-operator.v0.3.0-alpha.2
    uid: d2980a47-3adf-40d4-a5de-431fd30b620d
  resourceVersion: "1691626"
  selfLink: /apis/rbac.authorization.k8s.io/v1/namespaces/test-prom2/roles/prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
  uid: 7bd450e5-4c5e-497e-bf99-6e713276c31a
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
  - delete
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - ""
  resources:
  - secrets
  - pods
  - pods/exec
  - pods/log
  - pods
  - services
  - services/finalizers
  - endpoints
  - persistentvolumeclaims
  - events
  - configmaps
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - deployments
  - daemonsets
  - replicasets
  - statefulsets
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - monitoring.coreos.com
  resources:
  - servicemonitors
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - integreatly.org
  resources:
  - grafanadashboards
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - monitoring.3scale.net
  resources:
  - prometheusexporters
  - prometheusexporters/status
  - prometheusexporters/finalizers
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch

$ oc get rolebinding prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df -oyaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  creationTimestamp: "2021-03-12T17:38:06Z"
  labels:
    olm.copiedFrom: test-prom
    olm.owner: prometheus-exporter-operator.v0.3.0-alpha.2
    olm.owner.kind: ClusterServiceVersion
    olm.owner.namespace: test-prom2
  managedFields:
  - apiVersion: rbac.authorization.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:ownerReferences:
          k:{"uid":"b64d73e4-8699-47d5-9f9e-932089100155"}:
            .: {}
            f:apiVersion: {}
            f:blockOwnerDeletion: {}
            f:controller: {}
            f:kind: {}
            f:name: {}
            f:uid: {}
    manager: catalog
    operation: Update
    time: "2021-03-12T17:38:05Z"
  - apiVersion: rbac.authorization.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:olm.copiedFrom: {}
          f:olm.owner: {}
          f:olm.owner.kind: {}
          f:olm.owner.namespace: {}
        f:ownerReferences:
          .: {}
          k:{"uid":"d2980a47-3adf-40d4-a5de-431fd30b620d"}:
            .: {}
            f:apiVersion: {}
            f:blockOwnerDeletion: {}
            f:controller: {}
            f:kind: {}
            f:name: {}
            f:uid: {}
      f:roleRef:
        f:apiGroup: {}
        f:kind: {}
        f:name: {}
      f:subjects: {}
    manager: olm
    operation: Update
    time: "2021-03-12T17:38:06Z"
  name: prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
  namespace: test-prom2
  ownerReferences:
  - apiVersion: operators.coreos.com/v1alpha1
    blockOwnerDeletion: false
    controller: false
    kind: ClusterServiceVersion
    name: prometheus-exporter-operator.v0.3.0-alpha.2
    uid: d2980a47-3adf-40d4-a5de-431fd30b620d
  resourceVersion: "1691630"
  selfLink: /apis/rbac.authorization.k8s.io/v1/namespaces/test-prom2/rolebindings/prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
  uid: f3d169ad-12cb-41ae-a08a-a857acef02a7
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: prometheus-exporter-operator.v0.3.0-alpha.2-promethe-6ddc9849df
subjects:
- kind: ServiceAccount
  name: prometheus-exporter-operator-controller-manager
  namespace: test-prom

But every yaml can be found at branch upgrade-operator-sdk-1.4 (I won't delete this branch until this issue gets solved):

https://github.com/3scale/prometheus-exporter-operator/tree/upgrade-operator-sdk-1.4

@slopezz yeah the CSV has most of the information except for the actual values of the annotations that OLM is setting when it creates the Deployment which I'm curious to see

Sorry @fabianvf, regarding the CSV I just copy/pate the one from the repo, let me copy/paste the one deployed on the cluster (which has same WATCH_NAMESPACE value):

$ oc get csv prometheus-exporter-operator.v0.3.0-alpha.2 -oyaml
apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  annotations:
    alm-examples: |-
      [
        {
          "apiVersion": "monitoring.3scale.net/v1alpha1",
          "kind": "PrometheusExporter",
          "metadata": {
            "name": "example-memcached"
          },
          "spec": {
            "dbHost": "your-memcached-host",
            "dbPort": 11211,
            "grafanaDashboard": {
              "label": {
                "key": "autodiscovery",
                "value": "enabled"
              }
            },
            "type": "memcached"
          }
        }
      ]
    capabilities: Deep Insights
    categories: Monitoring
    certified: "false"
    containerImage: quay.io/3scale/prometheus-exporter-operator
    createdAt: "2020-06-08 00:00:00"
    description: Operator to setup 3rd party prometheus exporters, with a collection of grafana dashboards
    olm.operatorGroup: prometheus-exporter
    olm.operatorNamespace: test-prom
    olm.targetNamespaces: test-prom,test-prom2
    operatorframework.io/properties: '{"properties":[{"type":"olm.gvk","value":{"group":"monitoring.3scale.net","kind":"PrometheusExporter","version":"v1alpha1"}},{"type":"olm.package","value":{"packageName":"prometheus-exporter-operator","version":"0.3.0-alpha.2"}}]}'
    operators.operatorframework.io/builder: operator-sdk-v1.4.2
    operators.operatorframework.io/project_layout: ansible.sdk.operatorframework.io/v1
    repository: https://github.com/3scale/prometheus-exporter-operator
    support: Red Hat, Inc.
  creationTimestamp: "2021-03-12T17:38:05Z"
  generation: 1
  labels:
    olm.api.9d7f34c73b85de72: provided
    operators.coreos.com/prometheus-exporter-operator.test-prom: ""
  managedFields:
  - apiVersion: operators.coreos.com/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:alm-examples: {}
          f:capabilities: {}
          f:categories: {}
          f:certified: {}
          f:containerImage: {}
          f:createdAt: {}
          f:description: {}
          f:operatorframework.io/properties: {}
          f:operators.operatorframework.io/builder: {}
          f:operators.operatorframework.io/project_layout: {}
          f:repository: {}
          f:support: {}
      f:spec:
        .: {}
        f:apiservicedefinitions: {}
        f:customresourcedefinitions:
          .: {}
          f:owned: {}
        f:description: {}
        f:displayName: {}
        f:icon: {}
        f:install:
          .: {}
          f:spec:
            .: {}
            f:clusterPermissions: {}
            f:permissions: {}
          f:strategy: {}
        f:installModes: {}
        f:keywords: {}
        f:links: {}
        f:maintainers: {}
        f:maturity: {}
        f:provider:
          .: {}
          f:name: {}
          f:url: {}
        f:replaces: {}
        f:version: {}
    manager: catalog
    operation: Update
    time: "2021-03-12T17:38:05Z"
  - apiVersion: operators.coreos.com/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:olm.operatorGroup: {}
          f:olm.operatorNamespace: {}
          f:olm.targetNamespaces: {}
        f:labels:
          .: {}
          f:olm.api.9d7f34c73b85de72: {}
          f:operators.coreos.com/prometheus-exporter-operator.test-prom: {}
      f:spec:
        f:install:
          f:spec:
            f:deployments: {}
      f:status:
        .: {}
        f:conditions: {}
        f:lastTransitionTime: {}
        f:lastUpdateTime: {}
        f:message: {}
        f:phase: {}
        f:reason: {}
        f:requirementStatus: {}
    manager: olm
    operation: Update
    time: "2021-03-12T17:38:06Z"
  name: prometheus-exporter-operator.v0.3.0-alpha.2
  namespace: test-prom
  resourceVersion: "1694574"
  selfLink: /apis/operators.coreos.com/v1alpha1/namespaces/test-prom/clusterserviceversions/prometheus-exporter-operator.v0.3.0-alpha.2
  uid: b64d73e4-8699-47d5-9f9e-932089100155
spec:
  apiservicedefinitions: {}
  customresourcedefinitions:
    owned:
    - description: Configures a prometheus exporter to monitor a memcached instance
      displayName: PrometheusExporter
      kind: PrometheusExporter
      name: prometheusexporters.monitoring.3scale.net
      version: v1alpha1
  description: |
    A Kubernetes Operator based on the Operator SDK to centralize the setup of 3rd party prometheus exporters on **Kubernetes/OpenShift**, with a collection of grafana dashboards.

    You can setup different prometheus exporters to monitor the **internals from different databases**, **HTTP/TCP endpoints** (availability, latency, SSL/TLS certificate expiration...), or even any available **cloudwatch metric from any AWS Service**, by just providing a few parameters like **dbHost** or **dbPort** (operator manages the container image, port, argument, command, volumes... and also prometheus **ServiceMonitor** and **GrafanaDashboard** k8s objects).

    Current prometheus exporters types supported, managed by same prometheus-exporter-operator:
    * memcached
    * redis
    * mysql
    * postgresql
    * sphinx
    * es (elasticsearch)
    * cloudwatch
    * probe (blackbox)

    The operator manages the lifecycle of the following objects:
    * Deployment (one per CR)
    * Service (one per CR)
    * ServiceMonitor (optional, one per CR)
    * GrafanaDashboard (optional, one per Namespace)

    ### Documentation
    Documentation can be found on our [website](https://github.com/3scale/prometheus-exporter-operator#documentation).

    ### Getting help
    If you encounter any issues while using operator, you can create an issue on our [website](https://github.com/3scale/prometheus-exporter-operator) for bugs, enhancements, or other requests.

    ### Contributing
    You can contribute by:
    * Raising any issues you find using Prometheus Exporter Operator
    * Fixing issues by opening [Pull Requests](https://github.com/3scale/prometheus-exporter-operator/pulls)
    * Submitting a patch or opening a PR
    * Improving [documentation](https://github.com/3scale/prometheus-exporter-operator)
    * Talking about Prometheus Exporter Operator
    All bugs, tasks or enhancements are tracked as [GitHub issues](https://github.com/3scale/prometheus-exporter-operator/issues).

    ### License
    Prometheus Exporter Operator is licensed under the [Apache 2.0 license](https://github.com/3scale/prometheus-exporter-operator/blob/master/LICENSE)
  displayName: Prometheus Exporter Operator
  icon:
  - base64data: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxvbmU9Im5vIj8+CjxzdmcKICAgeG1sbnM6ZGM9Imh0dHA6Ly9wdXJsLm9yZy9kYy9lbGVtZW50cy8xLjEvIgogICB4bWxuczpjYz0iaHR0cDovL2NyZWF0aXZlY29tbW9ucy5vcmcvbnMjIgogICB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiCiAgIHhtbG5zOnN2Zz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciCiAgIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIKICAgeG1sbnM6c29kaXBvZGk9Imh0dHA6Ly9zb2RpcG9kaS5zb3VyY2Vmb3JnZS5uZXQvRFREL3NvZGlwb2RpLTAuZHRkIgogICB4bWxuczppbmtzY2FwZT0iaHR0cDovL3d3dy5pbmtzY2FwZS5vcmcvbmFtZXNwYWNlcy9pbmtzY2FwZSIKICAgaW5rc2NhcGU6dmVyc2lvbj0iMC45Mi40ICh1bmtub3duKSIKICAgaW5rc2NhcGU6ZXhwb3J0LXlkcGk9IjQxMC4xOSIKICAgaW5rc2NhcGU6ZXhwb3J0LXhkcGk9IjQxMC4xOSIKICAgaW5rc2NhcGU6ZXhwb3J0LWZpbGVuYW1lPSIvVXNlcnMvcmFlbC9Eb3dubG9hZHMvcHJvbWV0aGV1cy1leHBvcnRlci5wbmciCiAgIHNvZGlwb2RpOmRvY25hbWU9InByb21ldGhldXMtZXhwb3J0ZXItM3NjYWxlLW5ldy5zdmciCiAgIGlkPSJzdmc4NDMiCiAgIHByZXNlcnZlQXNwZWN0UmF0aW89InhNaWRZTWlkIG1lZXQiCiAgIHZpZXdCb3g9IjAgMCA3MzYuNDI1NzggNjE5Ljc0ODQ3IgogICBoZWlnaHQ9IjYxOS43NDg0N3B0IgogICB3aWR0aD0iNzM2LjQyNTc4cHQiCiAgIHZlcnNpb249IjEuMCI+CiAgPG1ldGFkYXRhCiAgICAgaWQ9Im1ldGFkYXRhODQ5Ij4KICAgIDxyZGY6UkRGPgogICAgICA8Y2M6V29yawogICAgICAgICByZGY6YWJvdXQ9IiI+CiAgICAgICAgPGRjOmZvcm1hdD5pbWFnZS9zdmcreG1sPC9kYzpmb3JtYXQ+CiAgICAgICAgPGRjOnR5cGUKICAgICAgICAgICByZGY6cmVzb3VyY2U9Imh0dHA6Ly9wdXJsLm9yZy9kYy9kY21pdHlwZS9TdGlsbEltYWdlIiAvPgogICAgICA8L2NjOldvcms+CiAgICA8L3JkZjpSREY+CiAgPC9tZXRhZGF0YT4KICA8ZGVmcwogICAgIGlkPSJkZWZzODQ3IiAvPgogIDxzb2RpcG9kaTpuYW1lZHZpZXcKICAgICBpbmtzY2FwZTpjdXJyZW50LWxheWVyPSJzdmc4NDMiCiAgICAgaW5rc2NhcGU6d2luZG93LW1heGltaXplZD0iMSIKICAgICBpbmtzY2FwZTp3aW5kb3cteT0iMjciCiAgICAgaW5rc2NhcGU6d2luZG93LXg9IjAiCiAgICAgaW5rc2NhcGU6Y3k9IjM1Ny4wNDcxNyIKICAgICBpbmtzY2FwZTpjeD0iMzc0LjM3OTciCiAgICAgaW5rc2NhcGU6em9vbT0iMC42ODc3ODIyNCIKICAgICBpbmtzY2FwZTpsb2NrZ3VpZGVzPSJ0cnVlIgogICAgIHNob3dncmlkPSJmYWxzZSIKICAgICBpZD0ibmFtZWR2aWV3ODQ1IgogICAgIGlua3NjYXBlOndpbmRvdy1oZWlnaHQ9IjEzNzYiCiAgICAgaW5rc2NhcGU6d2luZG93LXdpZHRoPSIyNTYwIgogICAgIGlua3NjYXBlOnBhZ2VzaGFkb3c9IjIiCiAgICAgaW5rc2NhcGU6cGFnZW9wYWNpdHk9IjAiCiAgICAgZ3VpZGV0b2xlcmFuY2U9IjEwIgogICAgIGdyaWR0b2xlcmFuY2U9IjEwIgogICAgIG9iamVjdHRvbGVyYW5jZT0iMTAiCiAgICAgYm9yZGVyb3BhY2l0eT0iMSIKICAgICBpbmtzY2FwZTpkb2N1bWVudC1yb3RhdGlvbj0iMCIKICAgICBib3JkZXJjb2xvcj0iIzY2NjY2NiIKICAgICBwYWdlY29sb3I9IiNmZmZmZmYiIC8+CiAgPHBhdGgKICAgICBkPSJtIDcxNS4xMDM4NiwzMS43NDg0NzYgYyAtMTEuNSw0LjkgLTI3LjMsMTEuNSAtMzUsMTQuNyAtMjAuNiw4LjcgLTUwLDIxLjEgLTY2LjUsMjguMSAtMTcuNSw3LjUgLTM4LjEsMTYuMiAtNzEuMywzMC4wMDAwMDQgLTEzLjYsNS43IC0yNC42LDEwLjcgLTI0LjUsMTEuMSAwLjIsMC4zIDEwLjcsOS40IDIzLjMsMjAuMSAxMi43LDEwLjggMjQuOSwyMS4zIDI3LjIsMjMuNCBsIDQuMywzLjkgLTYuNyw3LjUgYyAtMzEuOSwzNi4xIC04MS41LDg2LjkgLTExMi44LDExNS43IC0xMTAuNiwxMDEuNiAtMjIyLjQsMTcwLjIgLTMyNywyMDAuOCAtMTQuNyw0LjMgLTQxLjEwMDAwMSwxMC42IC00OS41MDAwMDEsMTEuOCAtMS45LDAuMyAtNi40LDEgLTEwLDEuNSAtMy42LDAuNiAtOS40LDEuNCAtMTMsMS43IC0zLjYsMC40IC05LjQsMSAtMTMsMS41IC0zLjYsMC40IC0xNC4yLDAuNyAtMjMuNywwLjYgLTkuNTAwMDAwMywtMC4xIC0xNy4xMDAwMDAzMiwwIC0xNi45MDAwMDAzMiwwLjEgMS40MDAwMDAwMiwxLjQgMzguMzAwMDAwMzIsNi45IDU2LjAwMDAwMDMyLDguMyAxMS41LDEgNTMuNTAwMDAxLDEuMyA2NC4xMDAwMDEsMC41IDIuOCwtMC4yIDkuMywtMC42IDE0LjUsLTEgNS4yLC0wLjMgMTAuNiwtMC44IDEyLC0xIDEuNCwtMC4yIDUuMiwtMC43IDguNSwtMSAzNy40LC0zLjkgODUuNiwtMTQuOSAxMjYuNSwtMjkuMSA5MS40LC0zMS42IDE3Ny4xLC04Mi45IDI2MSwtMTU2LjUgMjAuNSwtMTcuOSA2OC4xLC02NS4yIDg2LjQsLTg1LjcgbCAxMi44LC0xNC4yIDEuOCwyLjIgYyAwLjksMS4zIDEwLjgsMTQgMjEuOSwyOC4zIDExLjEsMTQuMyAyMC41LDI2LjEgMjEsMjYuMyAwLjQsMC4xIDEuNSwtMy43IDIuNSwtOC41IDAuOSwtNC45IDYuNCwtMzMuMSAxMi4xLC02Mi44IDUuNywtMjkuNyAxMi45LC02Ny4xIDE2LC04MyAxNywtODcuODAwMDA0IDE5LjYsLTEwMS43MDAwMDQgMTkuMywtMTAyLjkwMDAwNCAtMC4yLC0wLjggLTgsMiAtMjEuMyw3LjYgeiIKICAgICBpZD0icGF0aDgzNSIKICAgICBpbmtzY2FwZTpjb25uZWN0b3ItY3VydmF0dXJlPSIwIgogICAgIHN0eWxlPSJmaWxsOiM1YTVhNWE7ZmlsbC1vcGFjaXR5OjE7c3Ryb2tlOm5vbmU7c3Ryb2tlLXdpZHRoOjAuMSIgLz4KICA8cGF0aAogICAgIGQ9Im0gMjIuMjAzODU5LDE1Ni41NDg0OCBjIDIsMTUgMTkuNiw2OS42IDI1LDc3LjUgMC44LDEuMSAxLjQsMi41IDEuNCwzLjMgMCwwLjcgMiw1LjQgNC41LDEwLjQgMi41LDUgNC41LDkuNSA0LjUsOS45IDAsMS4xIDE1LjUsMjkuNCAyMyw0MS45IDkuMywxNS43IDM2LjEwMDAwMSw1NC44IDM4LjkwMDAwMSw1NyAwLjQsMC4zIDEuOCwyLjEgMy4xLDQgMS4zLDEuOSAyLjcsMy43IDMsNCAwLjMsMC4zIDIuNCwyLjcgNC41LDUuNSAyLjIsMi43IDQuMiw1LjIgNC41LDUuNSAwLjMsMC4zIDMuNCwzLjkgNyw4IDExLjgsMTMuNyAxNy4yLDE5LjMgMzguOCw0MC43IDExLjgsMTEuNyAyMi4xLDIxLjMgMjIuNywyMS4zIDEuNywwIDYzLjksLTM0LjcgNjQuMywtMzUuOCAwLjEsLTAuNSAtNC44LC00IC0xMSwtNy43IC04MS42LC00OS4zIC0xNTYuNjAwMDAxLC0xMTcgLTE5OC41MDAwMDEsLTE3OS41IC0xNS4zLC0yMi44IC0yNywtNDUuMyAtMzUuMiwtNjcuNSAtMSwtMi43IC0xLC0yLjYgLTAuNSwxLjUgeiIKICAgICBpZD0icGF0aDgzNyIKICAgICBpbmtzY2FwZTpjb25uZWN0b3ItY3VydmF0dXJlPSIwIgogICAgIHN0eWxlPSJmaWxsOiNlYzdhMDg7ZmlsbC1vcGFjaXR5OjE7c3Ryb2tlOm5vbmU7c3Ryb2tlLXdpZHRoOjAuMSIgLz4KICA8cGF0aAogICAgIGQ9Im0gMzE4LjMwMzg2LDQ3OS40NDg0OCBjIC0yNi44LDkuNyAtNDguNywxNy45IC00OC43LDE4LjMgMCwxLjIgMzEuNSwyMS42IDUyLjgsMzQuMiA1NCwzMiAxMjkuOSw2NiAxOTAuMiw4NS4xIGwgOC41LDIuNyAyNywtMC42IGMgMjUuOSwtMC42IDI3LjgsLTAuOCA0OCwtNC42IDQwLjUsLTcuNiA2OC43LC0xNy45IDg5LjIsLTMyLjYgNi4xLC00LjMgMTAuOSwtOC4xIDEwLjcsLTguNCAtMS40LC0xLjQgLTUyLjgsLTE2IC04OS40LC0yNS40IC0xNCwtMy43IC0yNi40LC03IC0yNy41LC03LjUgLTEuMSwtMC41IC02LjEsLTEuOCAtMTEuMSwtMyAtMTAsLTIuMyAtNjQuNCwtMjAuMyAtOTAuNywtMzAgLTMwLjgsLTExLjQgLTcyLjIsLTI4LjUgLTEwNC43LC00My40IGwgLTUuNiwtMi41IHoiCiAgICAgaWQ9InBhdGg4MzkiCiAgICAgaW5rc2NhcGU6Y29ubmVjdG9yLWN1cnZhdHVyZT0iMCIKICAgICBzdHlsZT0iZmlsbDojZWM3YTA4O2ZpbGwtb3BhY2l0eToxO3N0cm9rZTpub25lO3N0cm9rZS13aWR0aDowLjEiIC8+Cjwvc3ZnPgo=
    mediatype: image/svg+xml
  install:
    spec:
      clusterPermissions:
      - rules:
        - apiGroups:
          - authentication.k8s.io
          resources:
          - tokenreviews
          verbs:
          - create
        - apiGroups:
          - authorization.k8s.io
          resources:
          - subjectaccessreviews
          verbs:
          - create
        serviceAccountName: prometheus-exporter-operator-controller-manager
      deployments:
      - name: prometheus-exporter-operator-controller-manager
        spec:
          replicas: 1
          selector:
            matchLabels:
              control-plane: controller-manager
          strategy: {}
          template:
            metadata:
              creationTimestamp: null
              labels:
                control-plane: controller-manager
            spec:
              containers:
              - args:
                - --secure-listen-address=0.0.0.0:8443
                - --upstream=http://127.0.0.1:8080/
                - --logtostderr=true
                - --v=10
                image: gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0
                name: kube-rbac-proxy
                ports:
                - containerPort: 8443
                  name: https
                  protocol: TCP
                resources: {}
              - args:
                - --metrics-addr=127.0.0.1:8080
                - --enable-leader-election
                - --leader-election-id=prometheus-exporter-operator
                env:
                - name: ANSIBLE_GATHERING
                  value: explicit
                - name: WATCH_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.annotations['olm.targetNamespaces']
                image: quay.io/3scale/prometheus-exporter-operator:v0.3.0-alpha.2
                livenessProbe:
                  httpGet:
                    path: /readyz
                    port: 6789
                  initialDelaySeconds: 15
                  periodSeconds: 20
                  timeoutSeconds: 5
                name: manager
                readinessProbe:
                  httpGet:
                    path: /healthz
                    port: 6789
                  initialDelaySeconds: 5
                  periodSeconds: 10
                  timeoutSeconds: 5
                resources: {}
              serviceAccountName: prometheus-exporter-operator-controller-manager
              terminationGracePeriodSeconds: 10
      permissions:
      - rules:
        - apiGroups:
          - ""
          resources:
          - configmaps
          verbs:
          - get
          - list
          - watch
          - create
          - update
          - patch
          - delete
        - apiGroups:
          - ""
          resources:
          - events
          verbs:
          - create
          - patch
        - apiGroups:
          - ""
          resources:
          - secrets
          - pods
          - pods/exec
          - pods/log
          - pods
          - services
          - services/finalizers
          - endpoints
          - persistentvolumeclaims
          - events
          - configmaps
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - apps
          resources:
          - deployments
          - daemonsets
          - replicasets
          - statefulsets
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - ""
          resources:
          - services
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - ""
          resources:
          - secrets
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - ""
          resources:
          - configmaps
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - monitoring.coreos.com
          resources:
          - servicemonitors
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - integreatly.org
          resources:
          - grafanadashboards
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        - apiGroups:
          - monitoring.3scale.net
          resources:
          - prometheusexporters
          - prometheusexporters/status
          - prometheusexporters/finalizers
          verbs:
          - create
          - delete
          - get
          - list
          - patch
          - update
          - watch
        serviceAccountName: prometheus-exporter-operator-controller-manager
    strategy: deployment
  installModes:
  - supported: true
    type: OwnNamespace
  - supported: true
    type: SingleNamespace
  - supported: true
    type: MultiNamespace
  - supported: false
    type: AllNamespaces
  keywords:
  - monitoring
  - metrics
  - observability
  - prometheus
  - exporter
  - grafana
  - mysql
  - postgresql
  - memcached
  - redis
  - sphinx
  - elasticsearch
  - cloudwatch
  - blackbox
  links:
  - name: GitHub
    url: https://github.com/3scale/prometheus-exporter-operator
  maintainers:
  - email: 3scale-operations@redhat.com
    name: 3scale Ops
  maturity: alpha
  provider:
    name: Red Hat
    url: https://www.redhat.com
  replaces: prometheus-exporter-operator.v0.3.0-alpha.1
  version: 0.3.0-alpha.2
status:
  conditions:
  - lastTransitionTime: "2021-03-12T17:38:05Z"
    lastUpdateTime: "2021-03-12T17:38:05Z"
    message: requirements not yet checked
    phase: Pending
    reason: RequirementsUnknown
  - lastTransitionTime: "2021-03-12T17:38:05Z"
    lastUpdateTime: "2021-03-12T17:38:05Z"
    message: one or more requirements couldn't be found
    phase: Pending
    reason: RequirementsNotMet
  - lastTransitionTime: "2021-03-12T17:38:05Z"
    lastUpdateTime: "2021-03-12T17:38:05Z"
    message: all requirements found, attempting install
    phase: InstallReady
    reason: AllRequirementsMet
  - lastTransitionTime: "2021-03-12T17:38:06Z"
    lastUpdateTime: "2021-03-12T17:38:06Z"
    message: waiting for install components to report healthy
    phase: Installing
    reason: InstallSucceeded
  - lastTransitionTime: "2021-03-12T17:38:06Z"
    lastUpdateTime: "2021-03-12T17:38:06Z"
    message: |
      installing: waiting for deployment prometheus-exporter-operator-controller-manager to become ready: Waiting for deployment spec update to be observed...
    phase: Installing
    reason: InstallWaiting
  - lastTransitionTime: "2021-03-12T17:38:23Z"
    lastUpdateTime: "2021-03-12T17:38:23Z"
    message: install strategy completed with no errors
    phase: Succeeded
    reason: InstallSucceeded
  - lastTransitionTime: "2021-03-12T17:44:16Z"
    lastUpdateTime: "2021-03-12T17:44:16Z"
    message: |
      installing: waiting for deployment prometheus-exporter-operator-controller-manager to become ready: Waiting for deployment spec update to be observed...
    phase: Failed
    reason: ComponentUnhealthy
  - lastTransitionTime: "2021-03-12T17:44:16Z"
    lastUpdateTime: "2021-03-12T17:44:16Z"
    message: |
      installing: waiting for deployment prometheus-exporter-operator-controller-manager to become ready: Waiting for rollout to finish: 1 old replicas are pending termination...
    phase: Pending
    reason: NeedsReinstall
  - lastTransitionTime: "2021-03-12T17:44:16Z"
    lastUpdateTime: "2021-03-12T17:44:16Z"
    message: all requirements found, attempting install
    phase: InstallReady
    reason: AllRequirementsMet
  - lastTransitionTime: "2021-03-12T17:44:16Z"
    lastUpdateTime: "2021-03-12T17:44:16Z"
    message: waiting for install components to report healthy
    phase: Installing
    reason: InstallSucceeded
  - lastTransitionTime: "2021-03-12T17:44:16Z"
    lastUpdateTime: "2021-03-12T17:44:17Z"
    message: |
      installing: waiting for deployment prometheus-exporter-operator-controller-manager to become ready: Waiting for deployment spec update to be observed...
    phase: Installing
    reason: InstallWaiting
  - lastTransitionTime: "2021-03-12T17:44:17Z"
    lastUpdateTime: "2021-03-12T17:44:17Z"
    message: install strategy completed with no errors
    phase: Succeeded
    reason: InstallSucceeded
  - lastTransitionTime: "2021-03-12T17:46:12Z"
    lastUpdateTime: "2021-03-12T17:46:12Z"
    message: 'installing: deployment changed old hash=79bbc8454f, new hash=749b9fc645'
    phase: InstallReady
    reason: ComponentUnhealthy
  - lastTransitionTime: "2021-03-12T17:46:12Z"
    lastUpdateTime: "2021-03-12T17:46:12Z"
    message: waiting for install components to report healthy
    phase: Installing
    reason: InstallSucceeded
  - lastTransitionTime: "2021-03-12T17:46:12Z"
    lastUpdateTime: "2021-03-12T17:46:12Z"
    message: |
      installing: waiting for deployment prometheus-exporter-operator-controller-manager to become ready: Waiting for deployment spec update to be observed...
    phase: Installing
    reason: InstallWaiting
  - lastTransitionTime: "2021-03-12T17:46:26Z"
    lastUpdateTime: "2021-03-12T17:46:26Z"
    message: install strategy completed with no errors
    phase: Succeeded
    reason: InstallSucceeded
  lastTransitionTime: "2021-03-12T17:46:26Z"
  lastUpdateTime: "2021-03-12T17:46:26Z"
  message: install strategy completed with no errors
  phase: Succeeded
  reason: InstallSucceeded
  requirementStatus:
  - group: apiextensions.k8s.io
    kind: CustomResourceDefinition
    message: CRD is present and Established condition is true
    name: prometheusexporters.monitoring.3scale.net
    status: Present
    uuid: 9b0dc1b1-1253-42ef-b0f3-8603d7236772
    version: v1
  - dependents:
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["get","list","watch","create","update","patch","delete"],"apiGroups":[""],"resources":["configmaps"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["create","patch"],"apiGroups":[""],"resources":["events"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":[""],"resources":["secrets","pods","pods/exec","pods/log","pods","services","services/finalizers","endpoints","persistentvolumeclaims","events","configmaps"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":["apps"],"resources":["deployments","daemonsets","replicasets","statefulsets"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":[""],"resources":["services"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":[""],"resources":["secrets"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":[""],"resources":["configmaps"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":["monitoring.coreos.com"],"resources":["servicemonitors"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":["integreatly.org"],"resources":["grafanadashboards"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: namespaced rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":["monitoring.3scale.net"],"resources":["prometheusexporters","prometheusexporters/status","prometheusexporters/finalizers"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: cluster rule:{"verbs":["create"],"apiGroups":["authentication.k8s.io"],"resources":["tokenreviews"]}
      status: Satisfied
      version: v1
    - group: rbac.authorization.k8s.io
      kind: PolicyRule
      message: cluster rule:{"verbs":["create"],"apiGroups":["authorization.k8s.io"],"resources":["subjectaccessreviews"]}
      status: Satisfied
      version: v1
    group: ""
    kind: ServiceAccount
    message: ""
    name: prometheus-exporter-operator-controller-manager
    status: Present
    version: v1

The CSV provides the instructions to create a deployment to OLM, but there should be a deployment named prometheus-exporter-operator-controller-manager in the test-prom namespace, which is the actual deployment that is running your operator. I'm curious to see if that deployment that OLM created matches what we would expect based on the CSV + subscription

OK, here you have the deployed deployment:

$ oc get deployment prometheus-exporter-operator-controller-manager -oyaml
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "5"
  creationTimestamp: "2021-03-12T17:33:20Z"
  generation: 8
  labels:
    olm.deployment-spec-hash: 749b9fc645
    olm.owner: prometheus-exporter-operator.v0.3.0-alpha.2
    olm.owner.kind: ClusterServiceVersion
    olm.owner.namespace: test-prom
    operators.coreos.com/prometheus-exporter-operator.test-prom: ""
  managedFields:
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:olm.deployment-spec-hash: {}
          f:olm.owner: {}
          f:olm.owner.kind: {}
          f:olm.owner.namespace: {}
          f:operators.coreos.com/prometheus-exporter-operator.test-prom: {}
        f:ownerReferences:
          .: {}
          k:{"uid":"b64d73e4-8699-47d5-9f9e-932089100155"}:
            .: {}
            f:apiVersion: {}
            f:blockOwnerDeletion: {}
            f:controller: {}
            f:kind: {}
            f:name: {}
            f:uid: {}
      f:spec:
        f:progressDeadlineSeconds: {}
        f:replicas: {}
        f:revisionHistoryLimit: {}
        f:selector: {}
        f:strategy:
          f:rollingUpdate:
            .: {}
            f:maxSurge: {}
            f:maxUnavailable: {}
          f:type: {}
        f:template:
          f:metadata:
            f:annotations:
              .: {}
              f:alm-examples: {}
              f:capabilities: {}
              f:categories: {}
              f:certified: {}
              f:containerImage: {}
              f:createdAt: {}
              f:description: {}
              f:olm.operatorGroup: {}
              f:olm.operatorNamespace: {}
              f:olm.targetNamespaces: {}
              f:operatorframework.io/properties: {}
              f:operators.operatorframework.io/builder: {}
              f:operators.operatorframework.io/project_layout: {}
              f:repository: {}
              f:support: {}
            f:labels:
              .: {}
              f:control-plane: {}
          f:spec:
            f:containers:
              k:{"name":"kube-rbac-proxy"}:
                .: {}
                f:args: {}
                f:env:
                  .: {}
                  k:{"name":"ANSIBLE_DEBUG_LOGS"}:
                    .: {}
                    f:name: {}
                    f:value: {}
                  k:{"name":"OPERATOR_CONDITION_NAME"}:
                    .: {}
                    f:name: {}
                    f:value: {}
                f:image: {}
                f:imagePullPolicy: {}
                f:name: {}
                f:ports:
                  .: {}
                  k:{"containerPort":8443,"protocol":"TCP"}:
                    .: {}
                    f:containerPort: {}
                    f:name: {}
                    f:protocol: {}
                f:resources: {}
                f:terminationMessagePath: {}
                f:terminationMessagePolicy: {}
              k:{"name":"manager"}:
                .: {}
                f:args: {}
                f:env:
                  .: {}
                  k:{"name":"ANSIBLE_DEBUG_LOGS"}:
                    .: {}
                    f:name: {}
                    f:value: {}
                  k:{"name":"ANSIBLE_GATHERING"}:
                    .: {}
                    f:name: {}
                    f:value: {}
                  k:{"name":"OPERATOR_CONDITION_NAME"}:
                    .: {}
                    f:name: {}
                    f:value: {}
                  k:{"name":"WATCH_NAMESPACE"}:
                    .: {}
                    f:name: {}
                    f:valueFrom:
                      .: {}
                      f:fieldRef:
                        .: {}
                        f:apiVersion: {}
                        f:fieldPath: {}
                f:image: {}
                f:imagePullPolicy: {}
                f:livenessProbe:
                  .: {}
                  f:failureThreshold: {}
                  f:httpGet:
                    .: {}
                    f:path: {}
                    f:port: {}
                    f:scheme: {}
                  f:initialDelaySeconds: {}
                  f:periodSeconds: {}
                  f:successThreshold: {}
                  f:timeoutSeconds: {}
                f:name: {}
                f:readinessProbe:
                  .: {}
                  f:failureThreshold: {}
                  f:httpGet:
                    .: {}
                    f:path: {}
                    f:port: {}
                    f:scheme: {}
                  f:initialDelaySeconds: {}
                  f:periodSeconds: {}
                  f:successThreshold: {}
                  f:timeoutSeconds: {}
                f:resources: {}
                f:terminationMessagePath: {}
                f:terminationMessagePolicy: {}
            f:dnsPolicy: {}
            f:restartPolicy: {}
            f:schedulerName: {}
            f:securityContext: {}
            f:serviceAccount: {}
            f:serviceAccountName: {}
            f:terminationGracePeriodSeconds: {}
    manager: olm
    operation: Update
    time: "2021-03-12T17:46:12Z"
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:deployment.kubernetes.io/revision: {}
      f:status:
        f:availableReplicas: {}
        f:conditions:
          k:{"type":"Progressing"}:
            f:lastUpdateTime: {}
            f:message: {}
            f:reason: {}
        f:observedGeneration: {}
        f:readyReplicas: {}
        f:replicas: {}
    manager: kube-controller-manager
    operation: Update
    time: "2021-03-12T17:46:26Z"
  name: prometheus-exporter-operator-controller-manager
  namespace: test-prom
  ownerReferences:
  - apiVersion: operators.coreos.com/v1alpha1
    blockOwnerDeletion: false
    controller: false
    kind: ClusterServiceVersion
    name: prometheus-exporter-operator.v0.3.0-alpha.2
    uid: b64d73e4-8699-47d5-9f9e-932089100155
  resourceVersion: "1694570"
  selfLink: /apis/apps/v1/namespaces/test-prom/deployments/prometheus-exporter-operator-controller-manager
  uid: 90e9afe7-cb13-4bfd-b551-2e8ba328541e
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      control-plane: controller-manager
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        alm-examples: |-
          [
            {
              "apiVersion": "monitoring.3scale.net/v1alpha1",
              "kind": "PrometheusExporter",
              "metadata": {
                "name": "example-memcached"
              },
              "spec": {
                "dbHost": "your-memcached-host",
                "dbPort": 11211,
                "grafanaDashboard": {
                  "label": {
                    "key": "autodiscovery",
                    "value": "enabled"
                  }
                },
                "type": "memcached"
              }
            }
          ]
        capabilities: Deep Insights
        categories: Monitoring
        certified: "false"
        containerImage: quay.io/3scale/prometheus-exporter-operator
        createdAt: "2020-06-08 00:00:00"
        description: Operator to setup 3rd party prometheus exporters, with a collection of grafana dashboards
        olm.operatorGroup: prometheus-exporter
        olm.operatorNamespace: test-prom
        olm.targetNamespaces: test-prom,test-prom2
        operatorframework.io/properties: '{"properties":[{"type":"olm.gvk","value":{"group":"monitoring.3scale.net","kind":"PrometheusExporter","version":"v1alpha1"}},{"type":"olm.package","value":{"packageName":"prometheus-exporter-operator","version":"0.3.0-alpha.2"}}]}'
        operators.operatorframework.io/builder: operator-sdk-v1.4.2
        operators.operatorframework.io/project_layout: ansible.sdk.operatorframework.io/v1
        repository: https://github.com/3scale/prometheus-exporter-operator
        support: Red Hat, Inc.
      creationTimestamp: null
      labels:
        control-plane: controller-manager
    spec:
      containers:
      - args:
        - --secure-listen-address=0.0.0.0:8443
        - --upstream=http://127.0.0.1:8080/
        - --logtostderr=true
        - --v=10
        env:
        - name: ANSIBLE_DEBUG_LOGS
          value: "true"
        - name: OPERATOR_CONDITION_NAME
          value: prometheus-exporter-operator.v0.3.0-alpha.2
        image: gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0
        imagePullPolicy: IfNotPresent
        name: kube-rbac-proxy
        ports:
        - containerPort: 8443
          name: https
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      - args:
        - --metrics-addr=127.0.0.1:8080
        - --enable-leader-election
        - --leader-election-id=prometheus-exporter-operator
        env:
        - name: ANSIBLE_GATHERING
          value: explicit
        - name: WATCH_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.annotations['olm.targetNamespaces']
        - name: ANSIBLE_DEBUG_LOGS
          value: "true"
        - name: OPERATOR_CONDITION_NAME
          value: prometheus-exporter-operator.v0.3.0-alpha.2
        image: quay.io/3scale/prometheus-exporter-operator:v0.3.0-alpha.2
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /readyz
            port: 6789
            scheme: HTTP
          initialDelaySeconds: 15
          periodSeconds: 20
          successThreshold: 1
          timeoutSeconds: 5
        name: manager
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 6789
            scheme: HTTP
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: prometheus-exporter-operator-controller-manager
      serviceAccountName: prometheus-exporter-operator-controller-manager
      terminationGracePeriodSeconds: 10
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2021-03-12T17:33:28Z"
    lastUpdateTime: "2021-03-12T17:33:28Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2021-03-12T17:33:20Z"
    lastUpdateTime: "2021-03-12T17:46:26Z"
    message: ReplicaSet "prometheus-exporter-operator-controller-manager-7444bbfd78" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 8
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

If I enter into the container, I can see correct value (the one from the OperatorGroup targetNamespaces):

$ kubectl exec -it prometheus-exporter-operator-controller-manager-7444bbfd78pzdlf -c manager -- /bin/bash
bash-4.4$ env | grep WATCH_NAMESPACE
WATCH_NAMESPACE=test-prom,test-prom2

Do you see anything wrong with the deployment @fabianvf ?

@slopezz Unfortunately I didn't see anything obvious (was hoping it would be as simple as a messed up annotation or something), I need to do a bit of reading into how multi-namespace RBAC is supposed to work as everything you've pasted looks like it should be working to me

Quick update: I have upgraded to operator-sdk v1.5.0 and it has same behaviour with multi namespace mode:

I create the OperatorGroup to 2 namespace, then create 2 different CRs on different namespaces, and they are reconciled and created all dependant objects
If I delete manually any dependant object on any of the CR namespaces, they are never recreated, operator doesnt notice they are missing
If I create the OperatorGroup to a sinle namespace, it works as expected and notice any missing dependant object immedtiately.

Maybe the problem is that there is no cache when using ansible on multinamespace?

operator-framework / operator-sdk