integr8ly / application-monitoring-operator

Operator for installing the Application Monitoring Stack on OpenShift (Prometheus, AlertManager, Grafana)
Apache License 2.0
30 stars 44 forks source link

500 Internal Error in Grafana and Prometheus UI with custom CA #138

Open biloussi opened 4 years ago

biloussi commented 4 years ago

Hi, I have deployed the application-monitoring-operator to a OpenShift 4.3 cluster built on-prem. All components seem to work fine, without errors. However, I get error "500 Internal Error" right after I try to login to Grafana UI or to Prometheus UI. We use internally signed certificates and custom Certification Authority in our OpenShift environment.

Steps to reproduce:

  1. I install application-monitoring-operator git clone https://github.com/integr8ly/application-monitoring-operator.git make cluster/install
    and wait for completion. Components install without errors.
  2. I go to routes and click a route to Grafana.
  3. I click "Login with OpenShift" and get redirected to the OpenShift login, then I insert my username and password, I authorize the application to read my information, I get "500 Internal Error" message on the page.
  4. Try steps 2 and 3 for Prometheus UI and also get "500 Internal Error".

Environment info: oc get pods NAME READY STATUS RESTARTS AGE alertmanager-application-monitoring-0 3/3 Running 0 59m application-monitoring-operator-5bc879f697-mcglx 1/1 Running 0 60m grafana-deployment-58746b4f54-hr4xs 2/2 Running 0 9m32s grafana-operator-66497b6fc6-q9lhc 1/1 Running 0 59m prometheus-application-monitoring-0 5/5 Running 1 59m prometheus-operator-76b4dfbb68-r7k95 1/1 Running 0 59m

Logs for grafana-proxy container:

2020/06/01 10:18:45 provider.go:117: Defaulting client-id to system:serviceaccount:application-monitoring:grafana-serviceaccount 2020/06/01 10:18:45 provider.go:122: Defaulting client-secret to service account token /var/run/secrets/kubernetes.io/serviceaccount/token 2020/06/01 10:18:45 provider.go:310: Delegation of authentication and authorization to OpenShift is enabled for bearer tokens and client certificates. 2020/06/01 10:18:45 oauthproxy.go:200: mapping path "/" => upstream "http://localhost:3000/" 2020/06/01 10:18:45 oauthproxy.go:221: compiled skip-auth-regex => "^/metrics" 2020/06/01 10:18:45 oauthproxy.go:227: OAuthProxy configured for Client ID: system:serviceaccount:application-monitoring:grafana-serviceaccount 2020/06/01 10:18:45 oauthproxy.go:237: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain: refresh:disabled 2020/06/01 10:18:45 http.go:106: HTTPS: listening on [::]:9091 2020/06/01 10:21:25 provider.go:392: authorizer reason: 2020/06/01 10:21:28 provider.go:573: Performing OAuth discovery against https://172.30.0.1/.well-known/oauth-authorization-server 2020/06/01 10:21:28 provider.go:613: 200 GET https://172.30.0.1/.well-known/oauth-authorization-server { "issuer": "https://oauth-openshift.apps.os4-test.lab.local", "authorization_endpoint": "https://oauth-openshift.apps.os4-test.lab.local/oauth/authorize", "token_endpoint": "https://oauth-openshift.apps.os4-test.lab.local/oauth/token", "scopes_supported": [ "user:check-access", "user:full", "user:info", "user:list-projects", "user:list-scoped-projects" ], "response_types_supported": [ "code", "token" ], "grant_types_supported": [ "authorization_code", "implicit" ], "code_challenge_methods_supported": [ "plain", "S256" ] } 2020/06/01 10:21:38 provider.go:573: Performing OAuth discovery against https://172.30.0.1/.well-known/oauth-authorization-server 2020/06/01 10:21:38 provider.go:613: 200 GET https://172.30.0.1/.well-known/oauth-authorization-server { "issuer": "https://oauth-openshift.apps.os4-test.lab.local", "authorization_endpoint": "https://oauth-openshift.apps.os4-test.lab.local/oauth/authorize", "token_endpoint": "https://oauth-openshift.apps.os4-test.lab.local/oauth/token", "scopes_supported": [ "user:check-access", "user:full", "user:info", "user:list-projects", "user:list-scoped-projects" ], "response_types_supported": [ "code", "token" ], "grant_types_supported": [ "authorization_code", "implicit" ], "code_challenge_methods_supported": [ "plain", "S256" ] } 2020/06/01 10:21:38 oauthproxy.go:645: error redeeming code (client:10.254.3.1:47476): Post https://oauth-openshift.apps.os4-test.lab.local/oauth/token: x509: certificate signed by unknown authority 2020/06/01 10:21:38 oauthproxy.go:438: ErrorPage 500 Internal Error Internal Error

So I understand that the issue is with internally signed certificates.

I added two configmaps with our root certificates and labels: config.openshift.io/inject-trusted-cabundle: 'true'

I tried to add two sections to grafana-deployment: volumeMounts:

I also tried to edit Grafana and Prometheus instances of CRDs. However, in all cases the configuration is ovewritten by operator, which is expected behaviour, I believe.

Please advice, what is the correct flow of adding trusted-ca-bundle with this operator? Thank you! Sergiy