Storm Watch or Grid Charging distorts Self-Power Data

[jasonacox]

When a "Storm Watch" event occurs (or if you are allowed to do grid charging of your Powerwalls), the Grafana calculations for "Self-Power" and "Savings" will be wrong. This is due to the way that "from_solar" is computed:

SELECT (1+from_solar/abs(from_solar))*from_solar/2 FROM (SELECT integral(solar)/1000/3600 - integral(to_pw)/1000/3600 - integral(to_grid)/1000/3600 AS from_solar FROM autogen.http WHERE $timeFilter tz('America/Los_Angeles'))

This takes the total solar generation number (from_solar) and subtracts power to the grid (to_grid) and power to the Powerwalls (to_pw). Most of the time this works perfectly, but during events when "grid to Powerwall" occurs, the charging energy is reduced from the solar generation total.

I'm not sure there is an easy way to fix this. Ideally we would be able to decorate the "to_pw" (to Powerwall) with either "from Grid" or "from Solar" to be able to arrive at an accurate calculation.

[BJReplay]

I don't have an answer, but seeing this pop up in my email prompted me to think about the configuration interface when you log into your powerwall as installer (as I used to do when that was possible without toggling the switch on the side) to control charging mode, and go through the configuration wizard.

Typically (at least here in Australia, but I assume from reading the installation manuals, globally), they are configured with one CT for grid connection (per phase), and once CT for solar generation (per phase). Everything else reported from /api/meters/aggregates is derived from internal metering for the powerwall, and in particular home usage is a derived calculation based on site (the grid CT or CTs) based on the result of the net of solar, PW and site.

So this means that you should be able to re-derive the source of the data assuming it hasn't been lost between pypowerwall, telegraf and grafana.

Hopefully this has helped. Let me know if I can help (or test).

[BuongiornoTexas]

Edit: I realised this doesn't help with the underlying problem. It's just a different version of the solar integral. A reminder that I shouldn't post just before bed after a long day!

I'd suggest taking a closer look at differences in the cumulative energy exported from the solar aggregate meter. I'm still working through data gathering as part of #87 (delayed with a slow meter change over and minor roof damage from the solar install - discovered during a less than minor rain event), but it may be relevant here too.

On topic - while I was waiting for the meter changeover, I did some preliminary checks of inverter output vs solar output from the gateway cumulatives. Over a 20 day period, the inverters had an output of 693.1 kWh vs 692.4 as measured by the Tesla - so a difference of 0.7 kWh where I had a measurement error of up to 1kWh on one of the inverter meters, which I thought was not too bad!

Since the meter changeover, I've been using a more precise value from both inverters - +/- 0.1 kWh on each. With only 7 days of data, I have 278kWh generated with a difference of 0.1 kWh between the inverter meters and the Tesla reading.

I'd like more data to confirm, but at first glance it's looking as though this can be a reasonably accurate way to capture solar output.

[jasonacox]

That's great about the meter changeover!

I just realized that I can compute the "grid_to_pw" value with the rest of values we are gathering.

grid_to_pw = positive_values_only ( from_grid - home - solar )

SELECT solar, home, from_grid, from_pw, -to_pw, ((from_grid - home - solar) + ABS((from_grid - home - solar))) / 2 AS grid_to_pw FROM autogen.http WHERE $timeFilter tz('America/Los_Angeles')

The resulting show the charge from grid event:

Showing only the "grid_to_pw" flows:

Now what to do with that to fix the "Self Power Data"! 🤣

[youzer-name]

The self-powered issue is tricky. If you zoom into the period right after the grid_to_pw event, then you will see a bunch of energy coming from the battery and you'll be counting that as 'From Powerwall" which I think generally comes with an assumption that energy from the Powerwall is NOT grid power. So the next day you might show nothing but "From Solar" and "From Powerwall" with zero grid use, but that isn't really accurate. I don't see any reasonable way to work around that.

I do like the grid_to_pw in red... I'm going to add that to my usage panels, but I think the zero (or maybe near-zero) values need to be hidden because it put a red line across the entire chart.

[youzer-name]

I had been using mean values with a group by time($__interval), so I added that to the query. I also noticed that the query you posted didn't show any power going to the grid, so I changed the "Grid Usage" part to be:

mean(from_grid) - mean(to_grid") as grid

The whole query is:

SELECT  mean(solar) as solar , mean(home) as home, mean(from_grid) - mean(to_grid) as grid,
mean(from_pw) as from_pw, -mean(to_pw) as to_pw, 
((mean(from_grid) - mean(home) - mean(solar)) + ABS((mean(from_grid) - mean(home) - mean(solar)))) / 2  AS "grid_to_pw"
 FROM autogen.http WHERE $timeFilter group by time($__interval) fill(none)

Here's a stormwatch event followed by an outage back in July:

After that I decided to also show the power going to the Powerwall from the event in a different color by adding another query:

SELECT -((mean(from_grid) - mean(home) - mean(solar)) + ABS((mean(from_grid) - mean(home) - mean(solar)))) / 2  AS  pw_from_grid
from autogen.http where  (((from_grid - home - solar)  + ABS(from_grid - home -solar)) / 2) > 250
and  $timeFilter GROUP BY time($__interval) fill(null)

The ">250" is to filter out small values that aren't from a grid charging event.

I set a field override for this field to make it orange, not very transparent, and hide it in the legend. I also hid the "grid_to_pw" from the legend:

Applying the 'where >250' to both sides of the grid charging event has the effect of removing the red line at or near the zero point, so I separated this into 2 queries, one that gets the grid charging events (over 250 watts) and one that gets everything else:

SELECT  mean(solar) as solar, mean(home) as home, mean(from_grid) - mean(to_grid) as grid,
mean(from_pw) as from_pw, -mean(to_pw) as to_pw
 FROM autogen.http WHERE $timeFilter group by time($__interval) fill(none)

SELECT ((mean(from_grid) - mean(home) - mean(solar)) + ABS((mean(from_grid) - mean(home) - mean(solar)))) / 2  AS "grid_to_pw",
-((mean(from_grid) - mean(home) - mean(solar)) + ABS((mean(from_grid) - mean(home) - mean(solar)))) / 2  AS  pw_from_grid
  FROM autogen.http  WHERE
 (((from_grid - home - solar)  + ABS(from_grid - home -solar)) / 2) > 250
and  $timeFilter GROUP BY time($__interval) fill(null)

[BuongiornoTexas]

The self-powered issue is tricky. If you zoom into the period right after the grid_to_pw event, then you will see a bunch of energy coming from the battery and you'll be counting that as 'From Powerwall" which I think generally comes with an assumption that energy from the Powerwall is NOT grid power. So the next day you might show nothing but "From Solar" and "From Powerwall" with zero grid use, but that isn't really accurate. I don't see any reasonable way to work around that.

One possibility is to track the inflow, accumulation and withdrawal of grid energy and solar energy separately. But it leads to pretty ugly accounting.

[BJReplay]

Nice. Here's those ideas applied.

I've lost some other customisations (I hate Grafana, sometimes, lost multiple scales I had set up, temperature and so on).

First shows some unnecessary Storm Watch activations

Next shows some VPP FCAS activations in July, as well as some automation charging to ride through peak.

Firstly, over a week, (early morning spikes are clearly visible).

Then, the prior week with some power outages (I've got power status on the main chart as an On or Off trace).

Finally, zooming in on some automation based charging during the day to ride through peak on a low solar day including some early morning FCAS VPP dispatch, a power outage, and some automation charging.

(this section zoomed)

Those last two weeks were in July, so running when the automation was in charge preservation mode - it would switch to standby (set backup percentage to current SoC) at the end of peak to avoid discharging into off-peak load, only to have to charge again from off-peak grid load to discharge into peak, thereby avoiding round trip losses.

[jasonacox]

Thanks @BJReplay for the validation through many examples! And @youzer-name you are a Grafana wizard! I always learn some better way to visualize the data through your notes. Thank you!

On the "Self-Power" panel update... this panel is used to give you a percentage breakdown of what is powering your home. It is broken down into 3 buckets: Solar, Powerwall and Grid

Solar

The basic premise for computing "Self-Power" by solar is:

Solar_SelfPower = solar - to_pw - to_grid

This assumes that the only way to charge a Powerwall is via Solar, as such we remove any power sent to Powerwall or to Grid as they are not providing immediate "Self-Power" for the home. However, a miscalculation happens if the Powerwall is charged by the Grid. As discussed above, we can compute the power from Grid_to_powerwall and remove that from "Self-Power" by solar:

# Compute power from Grid to Powerwall 
grid_to_powerwall = ((from_grid - home - solar) + ABS((from_grid - home - solar))) / 2

# Compute power from Solar to Powerwall
solar_to_powerwall = to_pw - grid_to_powerwall

# Compute amount of Solar powering House
Solar_SelfPower = solar - solar_to_powerwall - to_grid

Using that equation to compute the Solar part of the "Self Power" panel looks like this:

SELECT (1+from_solar/abs(from_solar))*from_solar/2 FROM (SELECT integral(solar)/1000/3600 - (integral(to_pw) - integral(grid_to_pw))/1000/3600 - integral(to_grid)/1000/3600 AS from_solar FROM (SELECT from_solar, solar, to_pw, to_grid, ((from_grid - home - solar) + ABS((from_grid - home - solar))) / 2 AS grid_to_pw FROM autogen.http WHERE $timeFilter tz('America/Los_Angeles')))

Powerwall

The "From Powerwall" data is correct, showing the amount of energy the Powerwalls are providing to power the house. Actually, there is a condition where this breaks (see below).

Grid

The "From Grid" data which typically ONLY powers the home, also charges the Powerwall during these events. We need to remove that:

The "From Grid" query becomes:

# Compute amount of Grid powering House
Grid_SelfPower = from_grid - grid_to_powerwall 

SELECT (integral(from_grid) - integral(grid_to_pw))/1000/3600 AS from_grid FROM (SELECT from_grid, ((from_grid - home - solar) + ABS((from_grid - home - solar))) / 2 AS grid_to_pw FROM autogen.http WHERE $timeFilter tz('America/Los_Angeles'))

Example

Remaining Flaw

YES, there is still one major flaw (I'm suspect there are others), and that is, when the Powerwall feeds the Grid (Virtual Power Plant mode). When that occurs, the "From Powerwall" data will be wrong. We need a way to compute that and remove that from the "from_pw" data.