Closed kaykhancheckpoint closed 3 years ago
I Have exactly the same issue, default dashboards are not showing up in grafana.
I Have exactly the same issue, default dashboards are not showing up in grafana.
Did you find a solution yet?
if you go into your datasource section, check to see it's pointing the prometheus service endpoint i.e - prometheus-svc.blah-blahdomain:9090
I am encountering the same issues, prometheus and grafana deploys but default dashboards aren't available
Update: Deploying into the monitoring namespace fixes the issue
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.
This issue is being automatically closed due to inactivity.
I'm having the same issue.
I've installed the chart as follow:
helm install --namespace monitoring prometheus-operator prometheus-community/kube-prometheus-stack --set grafana.enabled=true
Data source in Grafana is not configured. If I manually add the URL (i.e. http://prometheus-operator-kube-p-prometheus.monitoring.svc.cluster.local:9090
) in the web UI then everything works.
How can I make sure the helm configures the datasource automatically?
Same thing here. No datasource is loaded on a fresh install. This used to work.
kubectl -n monitoring logs kube-prometheus-stack-grafana-54f5d666b9-8jwnb grafana-sc-datasources
shows
[2022-01-25 17:59:10] Starting collector
[2022-01-25 17:59:10] No folder annotation was provided, defaulting to k8s-sidecar-target-directory
[2022-01-25 17:59:10] Selected resource type: ('secret', 'configmap')
[2022-01-25 17:59:10] Loading incluster config ...
[2022-01-25 17:59:10] Config for cluster api at 'https://172.20.0.1:443' loaded...
[2022-01-25 17:59:10] Unique filenames will not be enforced.
[2022-01-25 17:59:10] 5xx response content will not be enabled.
[2022-01-25 17:59:15] Working on ADDED configmap monitoring/kube-prometheus-stack-grafana-datasource
[2022-01-25 17:59:15] POST request sent to http://localhost:3000/api/admin/provisioning/datasources/reload. Response: 401 Unauthorized {
"message": "Unauthorized"
}
I kept having Unauthorized
popups in Grafana, just like @CodechCFA's message.
Downgrading
Edit: no it did not.kube-prometheus-stack
to version 30.1.0
helped me.
We have the same problem. @CodechCFA @feber did you manage to find a solution?
For me it was my own mistake. I am not really sure how this problem was related to persistent storage. I was experimenting object storage as the backend of CSI, and used it as the persistent volume for both Prometheus and Grafana. Next, I changed the backend to block storage and the problem go away.
thanks @feber!
For us it turn out that the reason is we disable basic authentication (see https://github.com/grafana/helm-charts/pull/1046/). For now we set sidecar.skipReload
and sidecar.initDatasources
to true in grafana chart and it solved the problem, but when datasource config is updated, reloading grafana won't work though I think.
I'm having the same issue.
I've installed the chart as follow:
helm install --namespace monitoring prometheus-operator prometheus-community/kube-prometheus-stack --set grafana.enabled=true
Data source in Grafana is not configured. If I manually add the URL (i.e.
http://prometheus-operator-kube-p-prometheus.monitoring.svc.cluster.local:9090
) in the web UI then everything works.How can I make sure the helm configures the datasource automatically?
you can mention the url of prometheus service in the values.yaml section
eg: ## URL of prometheus datasource
# url: http://prometheus-stack-prometheus:9090/
in grafana sidecar section
I'm having the same issue. I've installed the chart as follow:
helm install --namespace monitoring prometheus-operator prometheus-community/kube-prometheus-stack --set grafana.enabled=true
Data source in Grafana is not configured. If I manually add the URL (i.e.http://prometheus-operator-kube-p-prometheus.monitoring.svc.cluster.local:9090
) in the web UI then everything works. How can I make sure the helm configures the datasource automatically?you can mention the url of prometheus service in the values.yaml section
eg: ## URL of prometheus datasource ## # url: http://prometheus-stack-prometheus:9090/
in grafana sidecar section
anyway it was old issue. for ref : we can pass things like this https://github.com/vishalk17/devops/blob/main/kubernetes/kubernetes-logging-observability/prometheus-grafana-loki/helm-installation/k8s-observability-prom-grafana-loki.sh#L36
Hi friends,
Ive recently installed kube-prometheus-stack on a fresh eks cluster, i expected there to be some default dashboards but open port forwarding grafana and logging in there appears to be none, there was also no datasource. I assume i would have these as grafana.defaultDashboardsEnabled is set to true
installation
version
side: If i create a new prometheus datasource on my own what should the url be?
values.yml