Migrate panels to the most recent version

[s-crypt]

On almost all the graphing panels grafana suggests migration for "more capability and better performance".

Would it be possible to update the default dashboard to contain these new panel charts?

Several of these are available to migrate with one click (and changing line width to 0 or fiddling for less than a minute to replicate the old chart). Most of the changes are from the panel types Graph (old) to Time series.

These are

• Grid Status
• Self Powered Doughnut Chart
• Home Usage
• Solar Energy
• From Powerwall
• Grid Usage
• String Voltage*
• String Current*
• String Power*
• Inverter Power*
• Powerwall Temps
• Frequency
• Voltages
• Powerwall Capacity
• Temp, Humid, Cloud*
• Wind, Pressure, Precip*

* = do not have data for and cannot verify a simple migration, but it's likely

There are also a couple panels where automatic migration is not good, but grafana still reccomends migration

• Grid Status
• Savings (complicatedish, as it needs to be changed to the bar chart type and it doesn't carry over settings well)
• Solar Energy Year

I have noticed an increase in performance (running on a raspberry pi) as opposed to the old chart type. I found it easy to migrate the chart types in the first list, but Grid Status, Savings, and Solar Energy Year stumped me as I am a grafana newbie.

[youzer-name]

I recall a discussion from last year where the same issue came up. I thought most of the panels had been updated, but that may only have been done in alternative dashboards and not the main project dashboard.

Here is a dashboard with my versions of Solar Energy Year and Grid Status that use current panels. You should be able to import the dashboard and then copy the panels over into your dashboard. My setup has diverged a bit from the main project, so some tweaking may be needed after importing the panels.

I don't have a Savings panel to share.

Dashboard JSON

{ "__inputs": [ { "name": "DS_INFLUXDB", "label": "InfluxDB", "description": "", "type": "datasource", "pluginId": "influxdb", "pluginName": "InfluxDB" } ], "__elements": { "XB8yzeRgk": { "name": "Grid Status", "uid": "XB8yzeRgk", "kind": 1, "model": { "alert": { "alertRuleTags": {}, "conditions": [ { "evaluator": { "params": [ 1 ], "type": "lt" }, "operator": { "type": "and" }, "query": { "params": [ "A", "5m", "now" ] }, "reducer": { "params": [], "type": "avg" }, "type": "query" } ], "executionErrorState": "alerting", "for": "5m", "frequency": "1m", "handler": 1, "message": "Grid Offline", "name": "Grid Status alert", "noDataState": "keep_state", "notifications": [] }, "datasource": { "type": "influxdb", "uid": "${DS_INFLUXDB}" }, "description": "", "fieldConfig": { "defaults": { "color": { "fixedColor": "green", "mode": "fixed" }, "custom": { "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "drawStyle": "line", "fillOpacity": 61, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "lineInterpolation": "linear", "lineWidth": 0, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "never", "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "links": [], "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null } ] }, "unit": "none" }, "overrides": [ { "matcher": { "id": "byName", "options": "off_grid" }, "properties": [ { "id": "color", "value": { "fixedColor": "red", "mode": "fixed" } }, { "id": "custom.hideFrom", "value": { "legend": true, "tooltip": true, "viz": false } }, { "id": "custom.axisPlacement", "value": "hidden" } ] }, { "matcher": { "id": "byName", "options": "Grid Status" }, "properties": [ { "id": "custom.axisPlacement", "value": "hidden" }, { "id": "mappings", "value": [ { "options": { "0": { "index": 1, "text": "Off Grid" }, "1": { "index": 0, "text": "On Grid" } }, "type": "value" } ] } ] } ] }, "hideTimeOverride": false, "interval": "", "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": false }, "tooltip": { "mode": "multi", "sort": "none" } }, "pluginVersion": "9.1.2", "targets": [ { "alias": "Grid Status", "datasource": { "type": "influxdb", "uid": "${DS_INFLUXDB}" }, "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "0" ], "type": "fill" } ], "measurement": "http", "orderByTime": "ASC", "policy": "raw", "query": "SELECT min(\"grid_status\") FROM \"grid\".\"http\" WHERE $timeFilter GROUP BY time($__interval) fill(previous) tz('America/New_York')", "rawQuery": true, "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "grid_status" ], "type": "field" }, { "params": [], "type": "sum" } ] ], "tags": [], "tz": "America/Los_Angeles" }, { "alias": "off_grid", "datasource": { "type": "influxdb", "uid": "${DS_INFLUXDB}" }, "hide": false, "query": "SELECT min(\"grid_status\") + 1 FROM \"grid\".\"http\" WHERE grid_status = 0 and $timeFilter GROUP BY time($__interval) fill(null) tz('America/New_York')", "rawQuery": true, "refId": "B", "resultFormat": "time_series" } ], "title": "Grid Status", "type": "timeseries" } } }, "__requires": [ { "type": "grafana", "id": "grafana", "name": "Grafana", "version": "9.1.2" }, { "type": "datasource", "id": "influxdb", "name": "InfluxDB", "version": "1.0.0" }, { "type": "panel", "id": "timeseries", "name": "Time series", "version": "" } ], "annotations": { "list": [ { "builtIn": 1, "datasource": { "type": "grafana", "uid": "-- Grafana --" }, "enable": true, "hide": true, "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", "target": { "limit": 100, "matchAny": false, "tags": [], "type": "dashboard" }, "type": "dashboard" } ] }, "editable": true, "fiscalYearStartMonth": 0, "graphTooltip": 0, "id": null, "links": [], "liveNow": false, "panels": [ { "gridPos": { "h": 4, "w": 12, "x": 0, "y": 0 }, "id": 4, "libraryPanel": { "uid": "XB8yzeRgk", "name": "Grid Status" } }, { "datasource": { "type": "influxdb", "uid": "${DS_INFLUXDB}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "drawStyle": "bars", "fillOpacity": 100, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "never", "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "links": [], "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null } ] }, "unit": "kwatth" }, "overrides": [ { "matcher": { "id": "byName", "options": "Solar" }, "properties": [ { "id": "color", "value": { "fixedColor": "yellow", "mode": "fixed" } } ] }, { "matcher": { "id": "byName", "options": "Home Usage" }, "properties": [ { "id": "color", "value": { "fixedColor": "#3274D9", "mode": "fixed" } }, { "id": "custom.drawStyle", "value": "line" }, { "id": "custom.fillOpacity", "value": 0 }, { "id": "custom.lineWidth", "value": 1 } ] } ] }, "gridPos": { "h": 8, "w": 24, "x": 0, "y": 4 }, "id": 2, "options": { "legend": { "calcs": [ "mean", "max" ], "displayMode": "table", "placement": "right", "showLegend": true }, "tooltip": { "mode": "multi", "sort": "none" } }, "pluginVersion": "9.1.2", "targets": [ { "alias": "Home Usage", "datasource": { "type": "influxdb", "uid": "${DS_INFLUXDB}" }, "groupBy": [ { "params": [ "1d" ], "type": "time" }, { "params": [ "null" ], "type": "fill" } ], "hide": false, "measurement": "http", "orderByTime": "ASC", "policy": "kwh", "query": "SELECT sum(\"home\") FROM \"kwh\".\"http\" WHERE (\"home\" > 0) GROUP BY time(1d) fill(null)", "rawQuery": true, "refId": "B", "resultFormat": "time_series", "select": [ [ { "params": [ "home" ], "type": "field" }, { "params": [], "type": "sum" } ] ], "tags": [ { "key": "home", "operator": ">", "value": "0" } ] }, { "alias": "Solar", "datasource": { "type": "influxdb", "uid": "${DS_INFLUXDB}" }, "groupBy": [ { "params": [ "1d" ], "type": "time" }, { "params": [ "null" ], "type": "fill" } ], "hide": false, "measurement": "http", "orderByTime": "ASC", "policy": "kwh", "query": "SELECT sum(\"solar\") FROM \"kwh\".\"http\" WHERE (\"solar\" > 0) GROUP BY time(1d) fill(0)", "rawQuery": true, "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "solar" ], "type": "field" }, { "params": [], "type": "sum" } ] ], "tags": [ { "key": "solar", "operator": ">", "value": "0" } ] } ], "timeFrom": "12M", "title": "Solar Energy Year", "type": "timeseries" } ], "schemaVersion": 37, "style": "dark", "tags": [], "templating": { "list": [] }, "time": { "from": "now-6h", "to": "now" }, "timepicker": {}, "timezone": "", "title": "Export", "uid": "gj8U6Omgk", "version": 2, "weekStart": "" }

[s-crypt]

Those panels work with no additional configuration! Thanks!

The solar energy year you provided did not have grid usage, so I added it, I also changed the lines to be staircase lines like the old graph.

Solar Energy Year JSON

{"id":74,"gridPos":{"h":8,"w":12,"x":0,"y":0},"type":"timeseries","title":"Solar Energy Year","datasource":{"type":"influxdb","uid":"TvWAC4iRk"},"pluginVersion":"9.1.2","timeFrom":"12M","libraryPanel":{"description":"","name":"Solar Energy Year","type":"timeseries","uid":"urTr7ciRk","version":5},"fieldConfig":{"defaults":{"custom":{"drawStyle":"bars","lineInterpolation":"linear","barAlignment":0,"lineWidth":1,"fillOpacity":100,"gradientMode":"none","spanNulls":false,"showPoints":"never","pointSize":5,"stacking":{"mode":"none","group":"A"},"axisPlacement":"auto","axisLabel":"","axisColorMode":"text","scaleDistribution":{"type":"linear"},"axisCenteredZero":false,"hideFrom":{"tooltip":false,"viz":false,"legend":false},"thresholdsStyle":{"mode":"off"}},"color":{"mode":"palette-classic"},"mappings":[],"thresholds":{"mode":"absolute","steps":[{"color":"green","value":null}]},"links":[],"unit":"kwatth"},"overrides":[{"matcher":{"id":"byName","options":"Solar"},"properties":[{"id":"color","value":{"fixedColor":"yellow","mode":"fixed"}}]},{"matcher":{"id":"byName","options":"Home Usage"},"properties":[{"id":"color","value":{"fixedColor":"#3274D9","mode":"fixed"}},{"id":"custom.drawStyle","value":"line"},{"id":"custom.fillOpacity","value":0},{"id":"custom.lineWidth","value":1},{"id":"custom.lineInterpolation","value":"stepAfter"}]},{"matcher":{"id":"byName","options":"Grid Usage"},"properties":[{"id":"color","value":{"fixedColor":"purple","mode":"fixed"}},{"id":"custom.lineWidth","value":1},{"id":"custom.lineInterpolation","value":"stepAfter"},{"id":"custom.drawStyle","value":"line"},{"id":"custom.fillOpacity","value":0}]}]},"options":{"tooltip":{"mode":"multi","sort":"none"},"legend":{"showLegend":true,"displayMode":"table","placement":"right","calcs":["mean","max"]}},"targets":[{"alias":"Home Usage","datasource":{"type":"influxdb","uid":"TvWAC4iRk"},"groupBy":[{"params":["1d"],"type":"time"},{"params":["null"],"type":"fill"}],"hide":false,"measurement":"http","orderByTime":"ASC","policy":"kwh","query":"SELECT sum(\"home\") FROM \"kwh\".\"http\" WHERE (\"home\" > 0) GROUP BY time(1d) fill(null)","rawQuery":true,"refId":"B","resultFormat":"time_series","select":[[{"params":["home"],"type":"field"},{"params":[],"type":"sum"}]],"tags":[{"key":"home","operator":">","value":"0"}]},{"alias":"Solar","datasource":{"type":"influxdb","uid":"TvWAC4iRk"},"groupBy":[{"params":["1d"],"type":"time"},{"params":["null"],"type":"fill"}],"hide":false,"measurement":"http","orderByTime":"ASC","policy":"kwh","query":"SELECT sum(\"solar\") FROM \"kwh\".\"http\" WHERE (\"solar\" > 0) GROUP BY time(1d) fill(0)","rawQuery":true,"refId":"A","resultFormat":"time_series","select":[[{"params":["solar"],"type":"field"},{"params":[],"type":"sum"}]],"tags":[{"key":"solar","operator":">","value":"0"}]},{"alias":"Grid Usage","datasource":{"type":"influxdb","uid":"TvWAC4iRk"},"hide":false,"query":"SELECT sum(\"from_grid\") - sum(\"to_grid\") as \"grid\" FROM \"kwh\".\"http\" WHERE $timeFilter GROUP BY time(1d) fill(0) tz('$tz')","rawQuery":true,"refId":"C","resultFormat":"time_series"}],"description":""}

I suppose all I would need is the savings panel and then I could make a pull request of the default dashboard with the new panels, though @jasonacox seems to be a grafana wizard so maybe he has some input.

[jasonacox]

Yes, I confess, I have been reluctant to do this for aesthetic reasons. We switched to the new time series graph for the main panel, but I found the grid lines to be annoyingly dim and untidy (bleed hanging lines without termination). I looked in to the code changes needed to adjust the grid lines and it isn't customizable (hardcoded) in Grafana. It is minor and if the majority of the community likes the new look we can update.

Old / New:

[s-crypt]

I knew it lol. I totally get the aesthetics reason (but weirdly I like the new grafana designs). Would you consider field testing a dashboard-new.json to see what people think and start a discussion post about it?

[jasonacox]

Yes of course! Feel free to submit a PR. I promise, I'll test drive it too. :)

[s-crypt]

I made a PR with most of the changes, except wind, pressure, and precip and savings in #295

[jasonacox]

Thanks @s-crypt ! I merged the change for anyone who wants to take a look it is here, dashboard-new.json.

There are a few minor issues or differences I would like to address:

• The "Solar Energy Year" graph needs to be set to transparent (the rest are).
• The "Monthly Analysis" graphs need a Y-axis minimum of 0, otherwise a 20kW day shows up as zero because everything else is > 20kW.
• The "Powerwall+" section is missing several graphs: "String Voltage", "String Current", "String Power", and "Inverter Power". Also "Voltages" and "Frequencies" should be move out into the Powerwall+ section (looks like the got moved to the "Alerts" section).

[s-crypt]

I think I accidentally shared my personal dashboard lol. Ill make a PR with the correct one.

[s-crypt]

In #297 I believe I fixed the issues you brought up. I did not set the grid usage to have a y=0 because that would only show power draw.

[jasonacox]

Thanks, @s-crypt ! Looks better!

On the y=0, for all the grpahs that should be under the "soft min" section to display without clipping:

I just added soft-min = 0:

Also, it looks like "Wind, Pressure and Precipitation" graph can be migrated but you will need to push "Pressure" to "axis placement" = right.

I'm not a fan of the new pie-chart (which is why I have the plugin for the old one) but I'm happy to keep it in there for a consistent "all new" look. 😉

[s-crypt]

Thanks for the information! Its nice to learn about grafana. I have submitted the following changes in #298

• made all monthly graphs have a soft y = 0
• center y axis for Grid Usage
• migrated Wind, Pressure, and Precip to time series
• changed Solar Energy Year to all line graphs with fill for consistency (and to overlap), adjusted solar to 40% opacity to better see home usage
• removed invalid overrides for Self Powered pie chart

[s-crypt]

Messing around with savings, there seems to be several issues when trying to convert savings to a bar chart. After changing many settings to replicate the first, there were some things that I could not figure out how to fix:

1) It sets the X axis to time with the query response time as the labels for the x axis 2) it shows only two lines for the axis 3) they are REALLY far apart and skinny.

Migrated Savings panel JSON

```JSON {"id":45,"gridPos":{"h":5,"w":9,"x":0,"y":17},"type":"barchart","title":"Savings","transformations":[],"datasource":{"type":"influxdb","uid":"TvWAC4iRk"},"pluginVersion":"9.1.2","hideTimeOverride":true,"description":"Estimated Savings","links":[],"fieldConfig":{"defaults":{"custom":{"lineWidth":0,"fillOpacity":100,"gradientMode":"none","axisPlacement":"auto","axisLabel":"","axisColorMode":"text","scaleDistribution":{"type":"linear"},"axisCenteredZero":false,"hideFrom":{"tooltip":false,"viz":false,"legend":false},"axisGridShow":true,"axisSoftMin":0},"color":{"mode":"thresholds"},"mappings":[],"thresholds":{"mode":"absolute","steps":[{"color":"green","value":null},{"color":"red","value":80}]},"links":[],"unit":"currencyUSD"},"overrides":[{"matcher":{"id":"byName","options":"powerwall > home"},"properties":[{"id":"color","value":{"fixedColor":"green","mode":"fixed"}}]},{"matcher":{"id":"byName","options":"solar > home"},"properties":[{"id":"color","value":{"fixedColor":"yellow","mode":"fixed"}}]},{"matcher":{"id":"byName","options":"solar > grid"},"properties":[{"id":"color","value":{"fixedColor":"purple","mode":"fixed"}}]},{"matcher":{"id":"byName","options":"total"},"properties":[{"id":"color","value":{"fixedColor":"blue","mode":"fixed"}}]}]},"interval":"","options":{"orientation":"auto","xTickLabelRotation":0,"xTickLabelSpacing":0,"showValue":"never","stacking":"normal","groupWidth":1,"barWidth":0.65,"barRadius":0,"tooltip":{"mode":"single","sort":"none"},"legend":{"showLegend":true,"displayMode":"table","placement":"right","calcs":["sum"]},"xTickLabelMaxLength":0},"targets":[{"alias":"powerwall > home","datasource":{"type":"influxdb","uid":"TvWAC4iRk"},"query":"SELECT fp FROM (SELECT sum(\"from_pw\") * $avg_buy_per_kwh AS fp FROM kwh.http WHERE $timeFilter tz('$tz'))","rawQuery":true,"refId":"A","resultFormat":"time_series"},{"alias":"solar > home","datasource":{"type":"influxdb","uid":"TvWAC4iRk"},"hide":false,"query":"SELECT s - (tp + tg) FROM (SELECT sum(\"solar\") * $avg_buy_per_kwh AS s, sum(\"to_pw\") * $avg_buy_per_kwh AS tp, sum(\"to_grid\") * $avg_buy_per_kwh AS tg FROM kwh.http WHERE $timeFilter tz('$tz'))\n","rawQuery":true,"refId":"B","resultFormat":"time_series"},{"alias":"solar > grid","datasource":{"type":"influxdb","uid":"TvWAC4iRk"},"hide":false,"query":"SELECT tgg FROM (SELECT sum(\"to_grid\") * $avg_sell_per_kwh AS tgg FROM kwh.http WHERE $timeFilter tz('$tz'))\n","rawQuery":true,"refId":"C","resultFormat":"time_series"},{"alias":"total","datasource":{"type":"influxdb","uid":"TvWAC4iRk"},"hide":false,"query":"SELECT tgg + (s - tp - tg ) + fp FROM (SELECT sum(\"to_pw\") * $avg_buy_per_kwh AS tp, sum(\"solar\") * $avg_buy_per_kwh AS s, sum(\"to_grid\") * $avg_buy_per_kwh AS tg, sum(\"from_pw\") * $avg_buy_per_kwh AS fp, sum(\"to_grid\") * $avg_sell_per_kwh AS tgg FROM kwh.http WHERE $timeFilter tz('$tz'))\n","rawQuery":true,"refId":"D","resultFormat":"time_series"}],"transparent":true} ```

[jasonacox]

Overall, I think the migration is an improvement for most of the panels. Thanks @s-crypt ! I for one, really appreciate your effort here.

I'm doing analysis of each to ensure we didn't lose anything in the migration before using this as our default dashboard for the project. There are some minor exceptions so far:

• The savings graph noted above doesn't work well yet in the new time series panel ( existing / new ):

• The "Self Powered" pie chart has been updated to continue to use the plugin. Others can optionally change it out for the "skinny tire" version.

• The "Current State" had some color changes that I am trying to figure out - I'll confirm they still meet the project color palette ( existing / new ):

• The "Grid Status" graph update (existing / new):

The new graph was missing the grid synchronization (-1) phase of grid status. I added that and also converted the dots to bars to create a more obvious off-grid signal:

[s-crypt]

Wow I didn't know it would be the default dashboard. That's cool!

Regarding your comments:

• The savings chart was changed to a bar chart, not time series. I do not know why there are times on the bottom. I would love to get it migrated but I am afraid that I am reaching my grafana knowledge limits and have tried several times to get the new panel to look like the old one with no success. EDIT: See next comment
• I changed some current state colors to better reflect their status (and admittedly I thought I was changing it on my own but I ended up changing dashboard-new. I am not good at juggling dashboards)
  • Battery was changed to orange to reflect discharging as red seems a bit aggressive, like an error
  • Solar turns grey when there is no generation, just like the idle for the grid.

[s-crypt]

I believe I was able to successfully migrate the savings panel to a bar chart in #314

Please let me know if you have any comments or questions! The PR also reverts the color changes in current state as those are a separate issue/comment.

[jasonacox]

I made 2 more changes to the default:

• Unlinked the "Grid Status" and "Solar Year" library panels. This will allow updates to those panels by future imports (otherwise Grafana will not replace old library panels)
• Updated "Grid Status" panel to remove annotation that would show up during a "grid down" event and made a cosmetic update to the display.

[s-crypt]

If you think it is appropriate, I have created #315 to migrate the savings panel on the default dashboard, along with fixing some sizing issues where the savings and self powered were too small on both default and new, and undid my janky tab spaced json for dashboard-new.

If you decide to merge that PR, I don't think that dashboard-new is necessary anymore, but I would love to hear what you think!

[jasonacox]

Great catch on the "estimated savings" calc miss (was using hard coded 0.19 instead of variables)!! I made that change to the default.

I'm not quite ready to accept the migrated savings graph. I need to tweak it a bit more to see why it bugs me. However, I do want to merge your dashboard-new.json changes. Can you remove the dashboard.json changes for now?

FYI - old / migrated savings:

[s-crypt]

Sounds good! I made those changes and a small fix in dashboard.json in #316

[jasonacox]

I'm a bit embarrassed that I'm so high maintenance 😊, but I like the row symmetry provided by h=5 rather than h=6 for the old graph:

h = 5

h = 6

[s-crypt]

alright, undid that change. I thought I shortened the height accidentally a while ago.