cncf / demo

Demo of CNCF technologies
https://cncf.io
Apache License 2.0
77 stars 39 forks source link

Influxdb-grafana addon #105

Closed namliz closed 7 years ago

namliz commented 7 years ago

Background

Influxdb is a timeseries data store. Grafana is a webapp that visualizes time series data and allows the creation of custom dashboards. It has support for several data sources, including influxdb (and more recently, Prometheus).

There are several slightly different versions of this pairing as a Kubernetes AddOn:

Neither is quite right as these examples combine what should be two separate deployments or replication controllers into one.

As a result the following race condition occurs:

1) Grafana container starts and is configured with a data source pointing to the influxdb service. 2) It fails to connect as the Influx pod is either not started or not ready yet. 3) As a result the dashboards are all blank with a warning marker. 4) The influxdb container starts.

At this point you can visit the Grafana UI and in the data source settings simply hit 'test connection'.

<img src="http://i.imgur.com/zDObMik.png" %50>

This forces a refresh and now data shows up in grafana and things work as expected.