search

msupply-foundation / msupply-dashboard

Home of the msupply dashboard, which is grafana with a couple of minor tweaks
1 stars 1 forks source link

Auto-backup all dashboards to github repo #91

Open CarlosNZ opened 4 years ago

CarlosNZ commented 4 years ago

Probably using this: https://github.com/ysde/grafana-backup-tool

CarlosNZ commented 4 years ago

Okay, have been having a look at this and I think it would be worth doing a thorough job of it, to achieve two things:

  1. Regular, automated backups of all dashboard installations in one browse-able and version-controlled location
  2. A easy-access reference point so see a summary of all dashboard installs, their structure and permissions, etc. at a glance.

To do this well will be a reasonable amount of work, so I'll put my proposed idea here to see if we think it's worth the time and effort.

I've had a play with the backup tool described above (https://github.com/ysde/grafana-backup-tool) -- it seems fine, but it's pretty basic and doesn't export things like user permissions, etc. We'd have to customise it quite substantially to get it to work how we want, so I think it's better to create a new node.js script from scratch. It's basically just making a bunch of GET requests to the Grafana API (https://grafana.com/docs/grafana/latest/http_api/), and the actual requests are fairly simple -- it's organising the backups, pushing changes to a git repo and parsing the info to a nice, summarised format that will take a bit of work.

So anyway, here's what I propose:

The script will download all of the following (as JSON exports) from each Dashboard installation, using the API:

(Basically, everything we'd need to re-install)

The data will be pushed to a (private) Github repo with the following folder structure:

.
|—— Name of Installation/Client
    |—— Summary.md      # Summarised Dashboard info
    |—— dashboards      # JSON export of all dashboards
        |—— <uid>.json
        |—— <uid>_versions.json
        |—— <uid>_permissions.json
        |—— <uid>.json
        |—— <uid>_versions.json
        |—— <uid>_permissions.json
        |—— ...
    |—— folders         # JSON exports of folders
        |—— folders.json
        |—— folder_<uid>_permissions.json
        |—— folder_<uid>_permissions.json
        |—— ...
    |—— datasources     # JSON export of datasources (i.e. postgres)
        |—— datasources.json    # (Doesn't store passwords)
    |—— users           # JSON export of Users and Teams
        |—— users.json
        |—— teams.json
    |—— alerts
        |—— [Notification channels and Alerts -- TBD, low priority]
    |—— preferences     # JSON export of preferences (theme/timezone, etc.)
        |—— preferences.json
|—— Eg. PNG
    |—— Summary.md
    |—— ...
    |—— ... etc.

Ideally, the script will extract any change messages from dashboards\<uid>_versions.json and use this as the git commit message, but will need to play around with it to see the most sensible way to do it (i.e. find a balance between lots and lots of meaningless commits vs informative commit info).

The script will then parse the relevant info out of each of those GET requests to create a markdown file — Summary.md — for each installation. Here is an example of how the PNG Summary.md might look:

Grafana Dashboard: PNG

URL: https://png.msupply.org:3000 Grafana version: 6.6.0

Overview

Production folder:

Develop folder:

Folder: Production

Permissions:

Dashboard: MOS Dashboard Critical/Essential/Normal Filter

Description: Shows MOS levels for store or item with various filters URL: https://png.msupply.org:3000/d/vS1FyMbZz/mos-dashboard-critical-essential-normal-filter Tags: Monitors mSupply Last edit: remove count panel by Sussoldash@sussol.net 2020-05-12 12:39:19 Permissions:

Template Variables (name/label):

Panel: Multi Store MOS for Single Item: $singleItem

Type: grafana-worldmap-panel Description: Shows stock levels for selected item(s) at the selected stores on a PNG map.

Panel: Multi Item Stock Levels across all facilities

Type: msupply-horizontal-bar Description: None

Panel: Multi Store MOS for Single Item: $singleItem

Type: msupply-horizontal-bar Description: None

Panel: Multi Item MOS for Single Store: $singleFacility

Type: msupply-horizontal-bar Description: None

Et Cetera for each Dashboard in each Folder...

Let me know your thoughts or any suggestions. Can make it a simpler (raw dumps shoved into folders), or a lot more sophisticated (interactive web front-end) -- just depends on what the priorities are.

Personally, I think having a summarised overview of each installation would be very useful. I was going to start a spreadsheet for this purpose when I realised that we could actually generate this information automatically as part of the automatic backup procedure.

I assume a node.js script would be the most straightforward way to go, but could easily do it with Python as well. Pros and cons?

Anyway, please discuss. :)

mark-prins commented 4 years ago

That would be awesome! All of the work would be unpaid, so we'd have to fit it in around other priorities, so I think start with the minimum and work up.

We are (or should be) (or soon will be) backing up the grafana.db for each instance, and can therefore use that to reinstall and/or extract permission and team information if we need to.

I'd say that the immediate need is

Would be ideal to also have the panels, though I'm worried that would be too much information. The panels are contained in the dashboard json, so it may be overkill to have them listed separately as well. It's just that it is hard to spot changes in SQL queries when comparing dashboard JSON diffs, the formatting is not easy to follow. The same problem exists with panels too, but you can see which ones have changes