Latest '1.0.2' tag won't install grafana operator on v3.11 Openshift

integr8ly / application-monitoring-operator

Operator for installing the Application Monitoring Stack on OpenShift (Prometheus, AlertManager, Grafana)

Apache License 2.0

30 stars 45 forks source link

Hi all,

we're trying to install the operator on a production v3.11 cluster and Grafana operator won't install. The make cluster/install goes fine, and all relevant CRDs are present

➜  application-monitoring-operator git:(master) oc project       
Using project "application-monitoring" on server "https://openshift-cluster.[DOMAIN]:8443"

➜  application-monitoring-operator git:(master)  oc get crds       
NAME                                                           CREATED AT
alertmanagers.monitoring.coreos.com                            2019-08-30T14:07:24Z
applicationmonitorings.applicationmonitoring.integreatly.org   2020-01-14T17:12:44Z
blackboxtargets.applicationmonitoring.integreatly.org          2020-01-14T17:12:46Z
bundlebindings.automationbroker.io                             2019-08-30T14:10:38Z
bundleinstances.automationbroker.io                            2019-08-30T14:10:38Z
bundles.automationbroker.io                                    2019-08-30T14:10:39Z
grafanadashboards.integreatly.org                              2020-01-14T17:12:47Z
grafanadatasources.integreatly.org                             2020-01-14T17:12:48Z
grafanas.integreatly.org                                       2020-01-14T17:12:46Z
podmonitors.monitoring.coreos.com                              2020-01-13T14:19:10Z
prometheuses.monitoring.coreos.com                             2019-08-30T14:07:24Z
prometheusrules.monitoring.coreos.com                          2019-08-30T14:07:24Z
servicemonitors.monitoring.coreos.com                          2019-08-30T14:07:24Z

➜  application-monitoring-operator git:(master) oc version
oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://openshift-cluster..[DOMAIN]::8443
openshift v3.11.135
kubernetes v1.11.0+d4cacc0
➜  application-monitoring-operator git:(master)

Other resources are correctly deployed

➜  application-monitoring-operator git:(master) oc get pods
NAME                                               READY     STATUS    RESTARTS   AGE
alertmanager-application-monitoring-0              3/3       Running   0          4m
application-monitoring-operator-749d9b6b54-mhj9s   1/1       Running   0          5m
prometheus-application-monitoring-0                5/5       Running   1          4m
prometheus-operator-86467cc6d8-l8cx4               1/1       Running   0          4m

We can see this error in application-monitoring-operator logs:

{"level":"info","ts":1579022331.038997,"logger":"controller_applicationmonitoring","msg":"Phase: Install GrafanaOperator"}
{"level":"info","ts":1579022331.0712292,"logger":"controller_applicationmonitoring","msg":"Error in InstallGrafanaOperator, resourceName=grafana-operator-role : err=error creating resource: roles.rbac.authorization.k8s.io \"grafana-operator-role\" is forbidden: attempt to grant extra privileges: [{[*] [integreatly.org] [grafanadashboards/status] [] []} {[*] [integreatly.org] [grafanadatasources/status] [] []} {[*] [integreatly.org] [grafanas/status] [] []}] user=&{system:serviceaccount:application-monitoring:application-monitoring-operator 14b6a2a5-36f1-11ea-a98e-005056920bc0 [system:serviceaccounts system:serviceaccounts:application-monitoring system:authenticated] map[]} ownerrules=[{[get] [ user.openshift.io] [users] [~] []} {[list] [ project.openshift.io] [projectrequests] [] []} {[get list] [ authorization.openshift.io] [clusterroles] [] []} {[get list watch] [rbac.authorization.k8s.io] [clusterroles] [] []} {[get list] [storage.k8s.io] [storageclasses] [] []} {[list watch] [ project.openshift.io] [projects] [] []} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[get] [] [] [] [/healthz /healthz/*]} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[list watch get] [servicecatalog.k8s.io] [clusterserviceclasses clusterserviceplans] [] []} {[get] [] [] [] [/healthz/ready]} {[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[create] [ build.openshift.io] [builds/docker builds/optimizeddocker] [] []} {[create] [ build.openshift.io] [builds/jenkinspipeline] [] []} {[create] [ build.openshift.io] [builds/source] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]} {[delete] [ oauth.openshift.io] [oauthaccesstokens oauthauthorizetokens] [] []} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[impersonate] [authentication.k8s.io] [userextras/scopes.authorization.openshift.io] [] []} {[create get] [ build.openshift.io] [buildconfigs/webhooks] [] []} {[*] [] [pods services services/finalizers endpoints persistentvolumeclaims events configmaps secrets serviceaccounts] [] []} {[*] [apps] [deployments deployments/finalizers daemonsets replicasets statefulsets] [] []} {[*] [monitoring.coreos.com] [alertmanagers prometheuses prometheusrules servicemonitors] [] []} {[*] [applicationmonitoring.integreatly.org] [applicationmonitorings applicationmonitorings/finalizers blackboxtargets blackboxtargets/finalizers] [] []} {[*] [integreatly.org] [grafanadatasources grafanadashboards grafanas grafanas/finalizers grafanadatasources/finalizers grafanadashboards/finalizers] [] []} {[*] [route.openshift.io] [routes routes/custom-host] [] []} {[*] [rbac.authorization.k8s.io] [rolebindings roles] [] []} {[*] [extensions] [ingresses] [] []} {[create] [authentication.k8s.io] [tokenreviews] [] []} {[create] [authorization.k8s.io] [subjectaccessreviews] [] []} {[get] [ image.openshift.io] [imagestreams/layers] [] []}] ruleResolutionErrors=[]"}
{"level":"error","ts":1579022331.0713165,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"applicationmonitoring-controller","request":"application-monitoring/example-applicationmonitoring","error":"error creating resource: roles.rbac.authorization.k8s.io \"grafana-operator-role\" is forbidden: attempt to grant extra privileges: [{[*] [integreatly.org] [grafanadashboards/status] [] []} {[*] [integreatly.org] [grafanadatasources/status] [] []} {[*] [integreatly.org] [grafanas/status] [] []}] user=&{system:serviceaccount:application-monitoring:application-monitoring-operator 14b6a2a5-36f1-11ea-a98e-005056920bc0 [system:serviceaccounts system:serviceaccounts:application-monitoring system:authenticated] map[]} ownerrules=[{[get] [ user.openshift.io] [users] [~] []} {[list] [ project.openshift.io] [projectrequests] [] []} {[get list] [ authorization.openshift.io] [clusterroles] [] []} {[get list watch] [rbac.authorization.k8s.io] [clusterroles] [] []} {[get list] [storage.k8s.io] [storageclasses] [] []} {[list watch] [ project.openshift.io] [projects] [] []} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[get] [] [] [] [/healthz /healthz/*]} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[list watch get] [servicecatalog.k8s.io] [clusterserviceclasses clusterserviceplans] [] []} {[get] [] [] [] [/healthz/ready]} {[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[create] [ build.openshift.io] [builds/docker builds/optimizeddocker] [] []} {[create] [ build.openshift.io] [builds/jenkinspipeline] [] []} {[create] [ build.openshift.io] [builds/source] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]} {[delete] [ oauth.openshift.io] [oauthaccesstokens oauthauthorizetokens] [] []} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[impersonate] [authentication.k8s.io] [userextras/scopes.authorization.openshift.io] [] []} {[create get] [ build.openshift.io] [buildconfigs/webhooks] [] []} {[*] [] [pods services services/finalizers endpoints persistentvolumeclaims events configmaps secrets serviceaccounts] [] []} {[*] [apps] [deployments deployments/finalizers daemonsets replicasets statefulsets] [] []} {[*] [monitoring.coreos.com] [alertmanagers prometheuses prometheusrules servicemonitors] [] []} {[*] [applicationmonitoring.integreatly.org] [applicationmonitorings applicationmonitorings/finalizers blackboxtargets blackboxtargets/finalizers] [] []} {[*] [integreatly.org] [grafanadatasources grafanadashboards grafanas grafanas/finalizers grafanadatasources/finalizers grafanadashboards/finalizers] [] []} {[*] [route.openshift.io] [routes routes/custom-host] [] []} {[*] [rbac.authorization.k8s.io] [rolebindings roles] [] []} {[*] [extensions] [ingresses] [] []} {[create] [authentication.k8s.io] [tokenreviews] [] []} {[create] [authorization.k8s.io] [subjectaccessreviews] [] []} {[get] [ image.openshift.io] [imagestreams/layers] [] []}] ruleResolutionErrors=[]","errorVerbose":"roles.rbac.authorization.k8s.io \"grafana-operator-role\" is forbidden: attempt to grant extra privileges: [{[*] [integreatly.org] [grafanadashboards/status] [] []} {[*] [integreatly.org] [grafanadatasources/status] [] []} {[*] [integreatly.org] [grafanas/status] [] []}] user=&{system:serviceaccount:application-monitoring:application-monitoring-operator 14b6a2a5-36f1-11ea-a98e-005056920bc0 [system:serviceaccounts system:serviceaccounts:application-monitoring system:authenticated] map[]} ownerrules=[{[get] [ user.openshift.io] [users] [~] []} {[list] [ project.openshift.io] [projectrequests] [] []} {[get list] [ authorization.openshift.io] [clusterroles] [] []} {[get list watch] [rbac.authorization.k8s.io] [clusterroles] [] []} {[get list] [storage.k8s.io] [storageclasses] [] []} {[list watch] [ project.openshift.io] [projects] [] []} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[get] [] [] [] [/healthz /healthz/*]} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[list watch get] [servicecatalog.k8s.io] [clusterserviceclasses clusterserviceplans] [] []} {[get] [] [] [] [/healthz/ready]} {[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[create] [ build.openshift.io] [builds/docker builds/optimizeddocker] [] []} {[create] [ build.openshift.io] [builds/jenkinspipeline] [] []} {[create] [ build.openshift.io] [builds/source] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]} {[delete] [ oauth.openshift.io] [oauthaccesstokens oauthauthorizetokens] [] []} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[impersonate] [authentication.k8s.io] [userextras/scopes.authorization.openshift.io] [] []} {[create get] [ build.openshift.io] [buildconfigs/webhooks] [] []} {[*] [] [pods services services/finalizers endpoints persistentvolumeclaims events configmaps secrets serviceaccounts] [] []} {[*] [apps] [deployments deployments/finalizers daemonsets replicasets statefulsets] [] []} {[*] [monitoring.coreos.com] [alertmanagers prometheuses prometheusrules servicemonitors] [] []} {[*] [applicationmonitoring.integreatly.org] [applicationmonitorings applicationmonitorings/finalizers blackboxtargets blackboxtargets/finalizers] [] []} {[*] [integreatly.org] [grafanadatasources grafanadashboards grafanas grafanas/finalizers grafanadatasources/finalizers grafanadashboards/finalizers] [] []} {[*] [route.openshift.io] [routes routes/custom-host] [] []} {[*] [rbac.authorization.k8s.io] [rolebindings roles] [] []} {[*] [extensions] [ingresses] [] []} {[create] [authentication.k8s.io] [tokenreviews] [] []} {[create] [authorization.k8s.io] [subjectaccessreviews] [] []} {[get] [ image.openshift.io] [imagestreams/layers] [] []}] ruleResolutionErrors=[]\nerror creating resource\ngithub.com/integr8ly/application-monitoring-operator/pkg/controller/applicationmonitoring.(*ReconcileApplicationMonitoring).createResource\n\tapplication-monitoring-operator/pkg/controller/applicationmonitoring/applicationmonitoring_controller.go:516\ngithub.com/integr8ly/application-monitoring-operator/pkg/controller/applicationmonitoring.(*ReconcileApplicationMonitoring).installGrafanaOperator\n\tapplication-monitoring-operator/pkg/controller/applicationmonitoring/applicationmonitoring_controller.go:468\ngithub.com/integr8ly/application-monitoring-operator/pkg/controller/applicationmonitoring.(*ReconcileApplicationMonitoring).Reconcile\n\tapplication-monitoring-operator/pkg/controller/applicationmonitoring/applicationmonitoring_controller.go:158\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tapplication-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:216\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tapplication-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:192\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tapplication-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:171\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tapplication-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tapplication-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\tapplication-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88\nruntime.goexit\n\t/home/dkirwan/bin/applications/go/src/runtime/asm_amd64.s:1357","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tapplication-monitoring-operator/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tapplication-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:218\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tapplication-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:192\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tapplication-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:171\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tapplication-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tapplication-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\tapplication-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}
W0114 17:18:51.623456       1 reflector.go:302] sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:204: watch of *v1.Secret ended with: The resourceVersion for the provided watch is too old.

Can anybody help? Thanks a lot

{"level":"info","ts":1579185611.6341345,"logger":"controller_applicationmonitoring","caller":"applicationmonitoring/applicationmonitoring_controller.go:465","msg":"Phase: Install GrafanaOperator"} {"level":"info","ts":1579185611.667897,"logger":"controller_applicationmonitoring","caller":"applicationmonitoring/applicationmonitoring_controller.go:469","msg":"Error in InstallGrafanaOperator, resourceName=grafana-operator-role : err=error creating resource: roles.rbac.authorization.k8s.io \"grafana-operator-role\" is forbidden: attempt to grant extra privileges: [{[*] [apps] [deployments/finalizers] [] []} {[*] [integreatly.org] [grafanadashboards/status] [] []} {[*] [integreatly.org] [grafanadatasources/status] [] []} {[*] [integreatly.org] [grafanas/status] [] []}] user=&{system:serviceaccount:application-monitoring:application-monitoring-operator 86fcbb4c-e431-11e9-a1c0-0050568cd90a [system:serviceaccounts system:serviceaccounts:application-monitoring system:authenticated] map[]} ownerrules=[{[get] [ user.openshift.io] [users] [~] []} {[list] [ project.openshift.io] [projectrequests] [] []} {[get list] [ authorization.openshift.io] [clusterroles] [] []} {[get list watch] [rbac.authorization.k8s.io] [clusterroles] [] []} {[get list] [storage.k8s.io] [storageclasses] [] []} {[list watch] [ project.openshift.io] [projects] [] []} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[get] [] [] [] [/healthz /healthz/*]} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[list watch get] [servicecatalog.k8s.io] [clusterserviceclasses clusterserviceplans] [] []} {[get] [] [] [] [/healthz/ready]} {[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[create] [ build.openshift.io] [builds/docker builds/optimizeddocker] [] []} {[create] [ build.openshift.io] [builds/jenkinspipeline] [] []} {[create] [ build.openshift.io] [builds/source] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]} {[delete] [ oauth.openshift.io] [oauthaccesstokens oauthauthorizetokens] [] []} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[impersonate] [authentication.k8s.io] [userextras/scopes.authorization.openshift.io] [] []} {[create get] [ build.openshift.io] [buildconfigs/webhooks] [] []} {[*] [] [pods services endpoints persistentvolumeclaims events configmaps secrets serviceaccounts] [] []} {[*] [apps] [deployments daemonsets replicasets statefulsets] [] []} {[*] [monitoring.coreos.com] [alertmanagers prometheuses prometheusrules servicemonitors] [] []} {[*] [applicationmonitoring.integreatly.org] [applicationmonitorings applicationmonitorings/finalizers blackboxtargets blackboxtargets/finalizers] [] []} {[*] [integreatly.org] [grafanadatasources grafanadashboards grafanas grafanas/finalizers grafanadatasources/finalizers grafanadashboards/finalizers] [] []} {[*] [route.openshift.io] [routes routes/custom-host] [] []} {[*] [rbac.authorization.k8s.io] [rolebindings roles] [] []} {[*] [extensions] [ingresses] [] []} {[create] [authentication.k8s.io] [tokenreviews] [] []} {[create] [authorization.k8s.io] [subjectaccessreviews] [] []} {[get] [ image.openshift.io] [imagestreams/layers] [] []}] ruleResolutionErrors=[]"} {"level":"error","ts":1579185611.6680799,"logger":"kubebuilder.controller","caller":"controller/controller.go:209","msg":"Reconciler error","Controller":"applicationmonitoring-controller","Request":"application-monitoring/example-applicationmonitoring","error":"error creating resource: roles.rbac.authorization.k8s.io \"grafana-operator-role\" is forbidden: attempt to grant extra privileges: [{[*] [apps] [deployments/finalizers] [] []} {[*] [integreatly.org] [grafanadashboards/status] [] []} {[*] [integreatly.org] [grafanadatasources/status] [] []} {[*] [integreatly.org] [grafanas/status] [] []}] user=&{system:serviceaccount:application-monitoring:application-monitoring-operator 86fcbb4c-e431-11e9-a1c0-0050568cd90a [system:serviceaccounts system:serviceaccounts:application-monitoring system:authenticated] map[]} ownerrules=[{[get] [ user.openshift.io] [users] [~] []} {[list] [ project.openshift.io] [projectrequests] [] []} {[get list] [ authorization.openshift.io] [clusterroles] [] []} {[get list watch] [rbac.authorization.k8s.io] [clusterroles] [] []} {[get list] [storage.k8s.io] [storageclasses] [] []} {[list watch] [ project.openshift.io] [projects] [] []} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[get] [] [] [] [/healthz /healthz/*]} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[list watch get] [servicecatalog.k8s.io] [clusterserviceclasses clusterserviceplans] [] []} {[get] [] [] [] [/healthz/ready]} {[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[create] [ build.openshift.io] [builds/docker builds/optimizeddocker] [] []} {[create] [ build.openshift.io] [builds/jenkinspipeline] [] []} {[create] [ build.openshift.io] [builds/source] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]} {[delete] [ oauth.openshift.io] [oauthaccesstokens oauthauthorizetokens] [] []} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[impersonate] [authentication.k8s.io] [userextras/scopes.authorization.openshift.io] [] []} {[create get] [ build.openshift.io] [buildconfigs/webhooks] [] []} {[*] [] [pods services endpoints persistentvolumeclaims events configmaps secrets serviceaccounts] [] []} {[*] [apps] [deployments daemonsets replicasets statefulsets] [] []} {[*] [monitoring.coreos.com] [alertmanagers prometheuses prometheusrules servicemonitors] [] []} {[*] [applicationmonitoring.integreatly.org] [applicationmonitorings applicationmonitorings/finalizers blackboxtargets blackboxtargets/finalizers] [] []} {[*] [integreatly.org] [grafanadatasources grafanadashboards grafanas grafanas/finalizers grafanadatasources/finalizers grafanadashboards/finalizers] [] []} {[*] [route.openshift.io] [routes routes/custom-host] [] []} {[*] [rbac.authorization.k8s.io] [rolebindings roles] [] []} {[*] [extensions] [ingresses] [] []} {[create] [authentication.k8s.io] [tokenreviews] [] []} {[create] [authorization.k8s.io] [subjectaccessreviews] [] []} {[get] [ image.openshift.io] [imagestreams/layers] [] []}] ruleResolutionErrors=[]","errorVerbose":"roles.rbac.authorization.k8s.io \"grafana-operator-role\" is forbidden: attempt to grant extra privileges: [{[*] [apps] [deployments/finalizers] [] []} {[*] [integreatly.org] [grafanadashboards/status] [] []} {[*] [integreatly.org] [grafanadatasources/status] [] []} {[*] [integreatly.org] [grafanas/status] [] []}] user=&{system:serviceaccount:application-monitoring:application-monitoring-operator 86fcbb4c-e431-11e9-a1c0-0050568cd90a [system:serviceaccounts system:serviceaccounts:application-monitoring system:authenticated] map[]} ownerrules=[{[get] [ user.openshift.io] [users] [~] []} {[list] [ project.openshift.io] [projectrequests] [] []} {[get list] [ authorization.openshift.io] [clusterroles] [] []} {[get list watch] [rbac.authorization.k8s.io] [clusterroles] [] []} {[get list] [storage.k8s.io] [storageclasses] [] []} {[list watch] [ project.openshift.io] [projects] [] []} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[get] [] [] [] [/healthz /healthz/*]} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[create] [ authorization.openshift.io] [selfsubjectrulesreviews] [] []} {[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[list watch get] [servicecatalog.k8s.io] [clusterserviceclasses clusterserviceplans] [] []} {[get] [] [] [] [/healthz/ready]} {[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[create] [ build.openshift.io] [builds/docker builds/optimizeddocker] [] []} {[create] [ build.openshift.io] [builds/jenkinspipeline] [] []} {[create] [ build.openshift.io] [builds/source] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]} {[delete] [ oauth.openshift.io] [oauthaccesstokens oauthauthorizetokens] [] []} {[get] [] [] [] [/version /version/* /api /api/* /apis /apis/* /oapi /oapi/* /openapi/v2 /swaggerapi /swaggerapi/* /swagger.json /swagger-2.0.0.pb-v1 /osapi /osapi/ /.well-known /.well-known/* /]} {[impersonate] [authentication.k8s.io] [userextras/scopes.authorization.openshift.io] [] []} {[create get] [ build.openshift.io] [buildconfigs/webhooks] [] []} {[*] [] [pods services endpoints persistentvolumeclaims events configmaps secrets serviceaccounts] [] []} {[*] [apps] [deployments daemonsets replicasets statefulsets] [] []} {[*] [monitoring.coreos.com] [alertmanagers prometheuses prometheusrules servicemonitors] [] []} {[*] [applicationmonitoring.integreatly.org] [applicationmonitorings applicationmonitorings/finalizers blackboxtargets blackboxtargets/finalizers] [] []} {[*] [integreatly.org] [grafanadatasources grafanadashboards grafanas grafanas/finalizers grafanadatasources/finalizers grafanadashboards/finalizers] [] []} {[*] [route.openshift.io] [routes routes/custom-host] [] []} {[*] [rbac.authorization.k8s.io] [rolebindings roles] [] []} {[*] [extensions] [ingresses] [] []} {[create] [authentication.k8s.io] [tokenreviews] [] []} {[create] [authorization.k8s.io] [subjectaccessreviews] [] []} {[get] [ image.openshift.io] [imagestreams/layers] [] []}] ruleResolutionErrors=[]\nerror creating resource\ngithub.com/integr8ly/application-monitoring-operator/pkg/controller/applicationmonitoring.(*ReconcileApplicationMonitoring).createResource\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/pkg/controller/applicationmonitoring/applicationmonitoring_controller.go:516\ngithub.com/integr8ly/application-monitoring-operator/pkg/controller/applicationmonitoring.(*ReconcileApplicationMonitoring).installGrafanaOperator\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/pkg/controller/applicationmonitoring/applicationmonitoring_controller.go:468\ngithub.com/integr8ly/application-monitoring-operator/pkg/controller/applicationmonitoring.(*ReconcileApplicationMonitoring).Reconcile\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/pkg/controller/applicationmonitoring/applicationmonitoring_controller.go:158\ngithub.com/integr8ly/application-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:207\ngithub.com/integr8ly/application-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:157\ngithub.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88\nruntime.goexit\n\t/home/travis/.gimme/versions/go1.10.8.linux.amd64/src/runtime/asm_amd64.s:2361","stacktrace":"github.com/integr8ly/application-monitoring-operator/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/integr8ly/application-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:209\ngithub.com/integr8ly/application-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:157\ngithub.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/home/travis/gopath/src/github.com/integr8ly/application-monitoring-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"} {"level":"info","ts":1579185612.6687596,"logger":"controller_applicationmonitoring","caller":"applicationmonitoring/applicationmonitoring_controller.go:115","msg":"Reconciling

➜ deploy git:(674aca8) oc get pods NAME READY STATUS RESTARTS AGE alertmanager-application-monitoring-0 3/3 Running 0 25m application-monitoring-operator-749d9b6b54-bkhrv 1/1 Running 0 19m grafana-deployment-6c4cb975b8-wswdq 2/2 Running 0 18m grafana-operator-66c44cc44c-sdnm7 1/1 Running 0 19m prometheus-application-monitoring-0 5/5 Running 1 25m prometheus-operator-86467cc6d8-lr6q5 1/1 Running 0 25m ➜ deploy git:(674aca8)

... ... - apiGroups: - integreatly.org attributeRestrictions: null resources: - grafanadashboards/status - grafanadatasources/status - grafanas/status verbs: - '*' ➜ ~ kgp -n application-monitoring NAME READY STATUS RESTARTS AGE alertmanager-application-monitoring-0 3/3 Running 0 19d application-monitoring-operator-749d9b6b54-php27 1/1 Running 0 2m grafana-deployment-6c4cb975b8-5kxpv 2/2 Running 0 2m grafana-operator-66c44cc44c-vmbzt 1/1 Running 0 2m prometheus-application-monitoring-0 5/5 Running 1 19d prometheus-operator-86467cc6d8-stvkw 1/1 Running 0 19d

integr8ly / application-monitoring-operator

Latest '1.0.2' tag won't install grafana operator on v3.11 Openshift #109