no data

[tirelibirefe]

Name and Version

bitnami/keycloak:7.1.7

What steps will reproduce the bug?

Hello, This is my metrics configuration:

metrics:
  enabled: true
  service:
    ports:
      http: 9990
    annotations:
      prometheus.io/scrape: "true"
      prometheus.io/port: "{{ .Values.metrics.service.ports.http }}"
  serviceMonitor:
    enabled: true
    namespace: "monitoring"
    interval: 30s
    scrapeTimeout: ""
    labels: {}
    selector: {}
    relabelings: []
    metricRelabelings: []
    honorLabels: false
    jobLabel: ""

I see Keycloak can be scraped by Prometheus:

I uploaded following Grafana dashboard https://grafana.com/grafana/dashboards/10441

What is the expected behavior?

I expect to see Prometheus data can be read by Grafana.

What do you see instead?

...but Grafana cannot detect variables, data cannot be read, all graphs are empty:

Additional information

Could you please advise?

Thanks & Regards

[pb82]

@tirelibirefe I believe you're using the wrong metrics URL. Your Prometheus config scrapes/metrics when it should look at /auth/realms/master/metrics

[tirelibirefe]

Hello @pb82 Thanks for your response.

My Keycloak installation/upgrade (helm) says:

...
You can access the Prometheus metrics following the steps below:

1. Get the Keycloak Prometheus metrics URL by running:
    kubectl port-forward --namespace keycloak svc/keycloak-metrics 9990:9990 &
    echo "Keycloak Prometheus metrics URL: http://127.0.0.1:9990/metrics"

2. Open a browser and access Keycloak Prometheus metrics using the obtained URL.

Also, when I curl /metrics, I get the output:

curl http://10.61.20.19:9990/metrics
...
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="clientSessions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="ejb",cache="http-remoting-connector"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="loginFailures"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="offlineClientSessions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="offlineSessions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="realmRevisions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="sessions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cac
...

Should I still need to change?

Thanks & Regards

[Tim-Schwalbe]

I have the same issue. The metrics mostly have none values, even when I trigger a specific event e.g. failed login. Can you maybe send a full list of your metrics from this endpoint?

[ivosh]

Hello @pb82 Thanks for your response.

My Keycloak installation/upgrade (helm) says:

...
You can access the Prometheus metrics following the steps below:

1. Get the Keycloak Prometheus metrics URL by running:
    kubectl port-forward --namespace keycloak svc/keycloak-metrics 9990:9990 &
    echo "Keycloak Prometheus metrics URL: http://127.0.0.1:9990/metrics"

2. Open a browser and access Keycloak Prometheus metrics using the obtained URL.

Also, when I curl /metrics, I get the output:

curl http://10.61.20.19:9990/metrics
...
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="clientSessions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="ejb",cache="http-remoting-connector"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="loginFailures"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="offlineClientSessions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="offlineSessions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="realmRevisions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cache="sessions"} 0.0
wildfly_infinispan_average_remove_time_seconds{cache_container="keycloak",cac
...

Should I still need to change?

Thanks & Regards

There are two kind of Prometheus metrics exposed:

• Wildfly (JBoss EAP) metrics on URL http(s)://<IP>:9990/metrics
• Keycloak metrics on URL http(s)://<IP>:8080/auth/realms/<realm>/metrics

I'd say you are interested in the latter.

[ivosh]

I have the same issue. The metrics mostly have none values, even when I trigger a specific event e.g. failed login. Can you maybe send a full list of your metrics from this endpoint?

Response is here: https://github.com/aerogear/keycloak-metrics-spi/issues/139#issuecomment-1450787973