search

grafana / grafana

The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
https://grafana.com
GNU Affero General Public License v3.0
61.15k stars 11.68k forks source link

Dashboards: Increased network traffic on dashboard loading caused by fetching alert rules for each panel #85147

Open edgarkz opened 2 months ago

edgarkz commented 2 months ago

What happened?

We are observing increased network traffic on dashboard loading caused by fetching rules?dashboard_uid for each panel we have with alerts attached - it seems like the rules api returns the whole alerting json which is quite heavy ~17mb per panel and therefore rendering of each panel takes a lot of time. image

What did you expect to happen?

rules?dashboard endpoint shouldn't be loaded so frequently - on slow network connections this makes dashboards unusable.

Did this work before?

We started with fresh grafana instance configured with PG backend db.

How do we reproduce it?

  1. have a dashboard with multiple panels and configured alerts
  2. reopen the dashboard and view 'developer tools' network tab
  3. for each panel query the alerts api loads the whole alerting configuration

Is the bug inside a dashboard panel?

No response

Environment (with versions)?

Grafana: 10.4 helm chart installation with below feature_toggles: enable: dashgpt,newVizTooltips,returnToPrevious,alertingSimplifiedRouting OS: macOS 14.4.1 Browser: Chrome 123

Grafana platform?

Kubernetes

Datasource(s)?

Prometheus 2.49.1

edgarkz commented 2 months ago

Added installation flags for [feature_toggles]: enable: dashgpt,newVizTooltips,returnToPrevious,alertingSimplifiedRouting Could be related to simplified alerting?

gbassinello commented 1 month ago

This is possibly related to https://github.com/grafana/grafana/issues/66701