Add real time monitoring

[jacobtomlinson]

After some discussion in pangeo-data/pangeo#184 and on our monthly catch up call yesterday I think it would be useful to add some default real time monitoring tools to this helm chart.

I see that there are likely to be two scenarios for people using this chart:

1. People who want a turn-key data analysis platform in the cloud and create a kubernetes cluster on a cloud compute platform purely for running Pangeo.
2. People who are already running a kubernetes cluster and want to install Pangeo on it.

My feeling is that the majority of Pangeo users will be in the first camp for now. Therefore it would make sense to include some real time monitoring tools such as prometheus and grafana along with some Pangeo specific default dashboard views.

To accommodate the second camp of users (which includes myself) this should be an optional config option as we already have monitoring on the cluster. But as it's likely that the majority will want it the default should be that is is enabled.

This would enable people to see useful information about the cluster like so:

[mrocklin]

+1 from me!

On Fri, Apr 6, 2018 at 3:26 AM, Jacob Tomlinson notifications@github.com wrote:

After some discussion in pangeo-data/pangeo#184 https://github.com/pangeo-data/pangeo/issues/184 and on our monthly catch up call yesterday I think it would be useful to add some default real time monitoring tools to this helm chart.

I see that there are likely to be two scenarios for people using this chart:

1. People who want a turn-key data analysis platform in the cloud and create a kubernetes cluster on a cloud compute platform purely for running Pangeo.
2. People who are already running a kubernetes cluster and want to install Pangeo on it.

My feeling is that the majority of Pangeo users will be in the first camp for now. Therefore it would make sense to include some real time monitoring tools such as prometheus https://github.com/kubernetes/charts/tree/master/stable/prometheus and grafana https://github.com/kubernetes/charts/tree/master/stable/grafana along with some Pangeo specific default dashboard views.

To accommodate the second camp of users (which includes myself) this should be an optional config option as we already have monitoring on the cluster. But as it's likely that the majority will want it the default should be that is is enabled.

This would enable people to see useful information about the cluster like so:

[image: image] https://user-images.githubusercontent.com/1610850/38408154-329354d0-3974-11e8-9043-57942ca06e70.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pangeo-data/helm-chart/issues/6, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszFaOxLnCL862TBnk0mhh5cYPIizgks5tlxirgaJpZM4TJnB3 .

[jhamman]

I spent some time looking into this today. Binderhub is using this approach for their diagnostic dashboards.

Their config is here: https://github.com/jupyterhub/mybinder.org-deploy/blob/d7a41ba5351141b5671a471bbc47f126e8d74852/mybinder/values.yaml#L184-L220

I didn't get deep enough to fully understand what we would need to make this happen but it seems fairly straightforward.

[guillaumeeb]

I think I've seen discussion on Grafana/Prometheus on other issues. Is this in place somewhere?

[guillaumeeb]

So should we try the same as what is done in the following PR? https://github.com/pangeo-data/pangeo-binder/pull/22

[jhamman]

@jacobtomlinson

To accommodate the second camp of users (which includes myself) this should be an optional config option as we already have monitoring on the cluster. But as it's likely that the majority will want it the default should be that is is enabled.

Do you have thoughts on how the configuration option should be handled?

[jacobtomlinson]

We should handle it the same way rbac is handled here.

Put in templates for the monitoring but wrap the whole thing in an if statement which checks a boolean flag in the config.

[guillaumeeb]

So must this be done in values.yaml, surrounded by the if statement, or must this go into template folder somehow?

Sorry, still learning Helm!

[jacobtomlinson]

There should be a new value in values.yaml something like

monitoring:
  enabled: true

Then we should add templates which add the monitoring resources and these should be wrapped in

{{- if .Values.monitoring.enabled -}}
...
{{- end -}}