Auto provision InfluxDB data source in Grafana

[longzheng]

Fixes https://github.com/jasonacox/Powerwall-Dashboard/discussions/491

Automatically set up the InfluxDB data source in Grafana.

This hard-codes the URL/port to the default 8086. In theory it might be possible to dynamically update this provisioning YML from INFLUXDB_PORTS in compose.env but it's not that straight forward.

I couldn't reproduce any issues with problems/timeouts as suggested in https://github.com/jasonacox/Powerwall-Dashboard/issues/461. My startup is equally fast with the provisioning file.

I saw these Grafana logs which aren't real errors, they're just indicating there's no provisioned plugins/notifiers/alerts.

2024-08-31 18:39:37 logger=provisioning.plugins t=2024-08-31T08:39:37.485272852Z level=error msg="Failed to read plugin provisioning files from directory" path=/var/lib/grafana/provisions/plugins error="open /var/lib/grafana/provisions/plugins: no such file or directory"
2024-08-31 18:39:37 logger=provisioning.notifiers t=2024-08-31T08:39:37.485794372Z level=error msg="Can't read alert notification provisioning files from directory" path=/var/lib/grafana/provisions/notifiers error="open /var/lib/grafana/provisions/notifiers: no such file or directory"
2024-08-31 18:39:37 logger=provisioning.alerting t=2024-08-31T08:39:37.4865298Z level=error msg="can't read alerting provisioning files from directory" path=/var/lib/grafana/provisions/alerting error="open /var/lib/grafana/provisions/alerting: no such file or directory"

[longzheng]

I tried to auto provision the "sun and moon" data source too but it involves parsing JSON with bash which isn't impossible but not too straight forward so I left that alone for now.

[jasonacox]

Thanks @longzheng ! I'll give it a test.

What instructions do we need to change in setup.sh or README?

I'll play around with the lat/long query to see if we can do that. It would be useful for weather too.

[jasonacox]

I think we can use this in setup.sh and have sun and moon auto provision. It will default to 0/0 and we can add instructions on how to change that data source.

# Get latitude and longitude 
LAT="0.0"
LONG="0.0"
PYTHON=$(command -v python3 || command -v python)
if [ -n "${PYTHON}" ]; then
    LAT=$(curl -s https://freeipapi.com/api/json | "${PYTHON}" -c "import sys, json; print(json.load(sys.stdin)['latitude'])")
    LONG=$(curl -s https://freeipapi.com/api/json | "${PYTHON}" -c "import sys, json; print(json.load(sys.stdin)['longitude'])")
fi

I thought about using jq but many OS variants won't have that installed. Python seems to be more likely.

[jasonacox]

Thanks for this great work @longzheng ! 🙏

I've merged and will be running tests to make sure we didn't break anything. Version v4.5.0.

[jasonacox]

Upgrade notes:

• The upgrade won't do anything new since the grafana.env already exists with provisioning commented out, which is fine. I manually uncommented it and the InfluxDB datasource was auto-provisioned without impacting anything.
• The start-up logs of grafana does show the errors of not finding other auto-provision items;

logger=provisioning.plugins t=2024-09-01T15:08:48.063660298Z level=error msg="Failed to read plugin provisioning files from directory" path=/var/lib/grafana/provisions/plugins error="open /var/lib/grafana/provisions/plugins: no such file or directory"
logger=provisioning.notifiers t=2024-09-01T15:08:48.064082276Z level=error msg="Can't read alert notification provisioning files from directory" path=/var/lib/grafana/provisions/notifiers error="open /var/lib/grafana/provisions/notifiers: no such file or directory"
logger=provisioning.alerting t=2024-09-01T15:08:48.064346108Z level=error msg="can't read alerting provisioning files from directory" path=/var/lib/grafana/provisions/alerting error="open /var/lib/grafana/provisions/alerting: no such file or directory"
logger=provisioning.dashboard t=2024-09-01T15:08:48.066779834Z level=error msg="can't read dashboard provisioning files from directory" path=/var/lib/grafana/provisions/dashboards error="open /var/lib/grafana/provisions/dashboards: no such file or directory"

[jasonacox]

New Install Test Notes:

• There is a bug in weather.sh setup that doesn't get the discovered longitude. I just pushed a hotfix for that.
• Setup worked perfectly and when I installed the dashboard.json, the auto provisioned datasources were available in the dropdown.

👍

[hulkster]

Nice work guys making things just that much easier for installs! ;-)

The Release Notes say "This adds an "(auto provisioned)" suffix to the data source name to prevent breaking exiting installations and to allow custom configurations" ... which I assume will apply to me since I'm a legacy user.

Since I try to go with defaults and minimize changes, might it be useful to have a quick "how-to" to convert my existing setup to simply use the auto-provisioned InfluxDB (and retain historical data) so that going forward, I'm more in sync with the default setup?

I could probably figure this out, but figure better to ask here since you guys are more familiar with the code and would come up with a more elegant/scalable solution.