Suggestion to Leverage Existing Monitoring Tools

[astrobokonon]

Terminology (taken from a Google Analytics help page): Dashboards are a collection of widgets that give you an overview of the reports and metrics you care about most. Dashboards let you monitor many metrics at once, so you can quickly check the health of your accounts or see correlations between different reports.

Using Grafana and influxdb I've created a monitoring framework to display in one place all the things I need to care about as an instrument scientist, and it can probably be adapted for more general use. There are two main components: the database on the back end, and the user interface/plotting on the front end. Keeping these split in two is central to the whole idea, since that allows the backend database to be used for other stuff too.

Even if this specific interface (Grafana) doesn't work, the database portion (influxdb) is almost certainly usable

At present, all the data is collected from the broker thru a single python program ("Iago" in https://github.com/LowellObservatory/DataServants) and the connection to the broker is managed hands-free. The program runs in perpetuity and monitors its own connection, reconnecting when necessary.

I settled on these particular for the following reasons

• No worries about PHP, Apache, or any of that web-stack kind of stuff. Grafana did the hard part of setting up all the various bits to make interactive plots.

• All components are put into docker containers (https://github.com/LowellObservatory/DockerDev/tree/master/DCTStack) which makes them super simple to start/stop/restart and almost trivial to upgrade and maintain.

• Since it's a web page, I can get to it (via the VPN) on my laptop/phone/tablet too.

• Data visualization is good and extendable via plugins.

• Plotting/interface is separate from the actual data, so you can get the data for other purposes without jumping through hoops

• Triggers and alerts are baked in and accessible via the GUI interface. This removes the need for most of the hand-crafted bash scripts currently scattered about.

• Open source tools, locally stored data

  • 

• Dashboards can have different views depending on the permissions/group of the user

Worth noting that the Mayall 4m is using basically the same setup for their engineering database purposes.

Some example screenshots - Instrument host computer health:

DCT status and history:

Instrument temperatures:

[dyerlytle]

I've uploaded a simple demo of a couple of interactive plots running Django with Bokeh plots. The files are in the "_proto/bokeh-graphs directory.

[astrobokonon]

I think we're ok to close this by now since we're well on the way to having a working prototype.