Cloudy days, solar clipping

[slopemad]

So I've been having a think about management of solar clipping, especially on cloudy days, where I want to export as much PV as I can (because of the rate I'm paid for PV, versus the rate I can subsequently buy power at from the grid.

My setup is particularly prone to clipping, because I have a Gen1 3.6 hybrid inverter, which can charge the battery at 2600, pass through the inverter at 3600, but I have 11 x 405W panels with a theoretical peak of about 4.5kWp (but I've been seeing 4800W today, and it's not even lunchtime). So I want to make sure that I'm passing all that extra solar into the battery to avoid losing it. It's also very intermittently cloudy, so the charger could be set to, say, 1250W to soak up all that spare solar into the battery, but when the clouds come over, I'm putting too much into the battery, when I'd be better off just exporting it. (Hey, why waste >10% on thermal losses!)

So, I've started writing a little AppDaemon app, at the moment it's just reading data and not doing anything with it as I'm trying to think of the best way to handle it. It's running this every 1 minute, collecting the last 5 minutes of data regarding PV generation and inverter throughput and processing it. This is the little routine:

    def collect_sensor_data(self, kwargs=None):
        # Calculate the start and end time for the data collection (last 5 minutes)
        end_time = datetime.datetime.now()
        start_time = end_time - datetime.timedelta(minutes=5)

        # Retrieve sensor data from Home Assistant
        pvgen_entity = "sensor.givtcp_saXXXXXXXX_pv_power"
        inverter_entity = "sensor.givtcp_saXXXXXXXX_invertor_power"
        forecast_now_entity = "sensor.solcast_pv_forecast_power_now"
        forecast_next_entity = "sensor.solcast_pv_forecast_power_next_30_mins"

        pvgen_data = self.get_history_async(pvgen_entity, start_time, end_time)
        inverter_data = self.get_history_async(inverter_entity, start_time, end_time)
        pvgen_values = [int(state['state']) for states in pvgen_data for state in states]
        inverter_values = [int(state['state']) for states in inverter_data for state in states]
        forecast_now_value = self.get_state(forecast_now_entity)
        forecast_next_value = self.get_state(forecast_next_entity)

        pvgen_min = min(pvgen_values)
        pvgen_max = max(pvgen_values)
        pvgen_avg = int(statistics.mean(pvgen_values))
        pvgen_stddev = int(statistics.stdev(pvgen_values))

        inverter_min = min(inverter_values)
        inverter_max = max(inverter_values)
        inverter_avg = int(statistics.mean(inverter_values))
        inverter_stddev = int(statistics.stdev(inverter_values))

        # Process collected sensor data (example: printing)
        self.log("PV Generation Data Collected from {} to {}: {}".format(start_time, end_time, pvgen_values))
        self.log("Inverter Data Collected from {} to {}: {}".format(start_time, end_time, inverter_values))
        self.log("PV Generation Statistics - Min: {}, Max: {}, Average: {}, Std Deviation: {}".format(pvgen_min, pvgen_max, pvgen_avg, pvgen_stddev))
        self.log("Inverter Statistics - Min: {}, Max: {}, Average: {}, Std Deviation: {}".format(inverter_min, inverter_max, inverter_avg, inverter_stddev))

        self.log("Solcast Forecast now: {}".format(forecast_now_value))
        self.log("Solcast Forecast next: {}".format(forecast_next_value))

It calls the get_history_async procedure which is just cut+paste from predbat (except it looks at and end time and start time rather than days.

The idea here being that I can capture the highest solar and inverter throughput measured in the last 5 mintues, and also look at averages and standard deviations (a high standard deviation means it's cloudy).

A typical output from this would be:

2024-03-28 10:37:18.047955 INFO PVCalc:: PV Generation Data Collected from 2024-03-28 10:32:17.939639 to 2024-03-28 10:37:17.939639: [4878, 1920, 1622, 1761, 1612, 1586, 2020, 2091, 3195, 1893, 1661, 1752, 1526, 1413]
2024-03-28 10:37:18.049386 INFO PVCalc:: Inverter Data Collected from 2024-03-28 10:32:17.939639 to 2024-03-28 10:37:17.939639: [3490, 1009, 1142, 1186, 1130, 1151, 1152, 1164, 1841, 1139, 1149, 1153, 1163, 1152]
2024-03-28 10:37:18.050675 INFO PVCalc:: PV Generation Statistics - Min: 1413, Max: 4878, Average: 2066, Std Deviation: 917
2024-03-28 10:37:18.051979 INFO PVCalc:: Inverter Statistics - Min: 1009, Max: 3490, Average: 1358, Std Deviation: 642
2024-03-28 10:37:18.055044 INFO PVCalc:: Solcast Forecast now: 2794
2024-03-28 10:37:18.057885 INFO PVCalc:: Solcast Forecast next: 3392

I can see from that, the solcast forecast suggests 2794W, the peak I've seen is 4898 (because my battery charge rate was set to about 1200), and it's pretty cloudy, which can be seen by the high standard deviation figure.

My theory here being that if this little app sees maximum inverter throughput at close to maximum, then I probably want to be increasing the battery charge rate to collect more spare solar. (and similarly, if peak inverter throughput is lower than it can actually handle, then we might want to reduce throughput.

The figures derived from this could also be used to establish the cloud model used by Predbat for the next hour or so, perhaps in a way that tapers from one established here to the one calculated using the solcast forecasts?

This is very much a work in progress that I'm looking at here, but I figured I'd share it here as it may well be of interest to integrate with Predbat and get more thoughts? (I think it should be a seperate .py file which runs every minute rather than every 5 minutes, to try and capture cloudy days, minimise clipping, and maximise export).

[gcoan]

There is logic to predict max inverter throughput rate based on setting inverter_limit in apps.yaml but I don't believe at the moment that Predbat does anything proactive with the battery charge level to minimise the impact of solar generation clipping. Its just full rate charging or zero rate charging.

I think there's different types of clipping that can occur:

• solar generation clipping (too much DC PV for the inverter)
• inverter throughput clipping (too much AC for the inverter)

The inverters can handle something like 50% extra DC I believe so its the latter in various forms is the more likely issue.

I've certainly started to see clipping happening in one of my own inverters. Array is 6.2kW on a 5kW gen 1 hybrid inverter. This morning ahead of the octopus power-up event from 11am Predbat was trying to discharge and export the battery, but that array was generating 4.7kW so the inverter was only able to discharge the battery at ~400W, resulting in the battery only getting down to 22% at the start of the power up event. If Predbat had been more aware that this could happen it could perhaps have started the discharge earlier.

The scenario you're highlighting @slopemad needs a different treatment, charging the battery at a lower rate so it trickle charges DC into the battery over the day and exports the remainder rather than the current 'fill up at full rate and export when full' which then can result in AC clipping.

More sophisticated understanding of clipping and dealing with it would be good in predbat.

An additional wrinkle for me having two arrays with different strings on different orientations is the solcast integration doesn't differentiate the different arrays.

[slopemad]

With the Hybrid inverters, the solar is connected to the DC side, and I can have up to 3.6kW through the inverter, and 2.6kW into the battery. For some reason it cannot be configured so that if you are clipping, it'll automatically put excess into the battery. Eco in GivEnergy means Solar through the inverter to supply house load, then Solar to the battery up to the charge limit, and then additional Solar through the inverter for export.

Obviously, if I allow the full 2600 into the battery, then it'll fill up early on sunny days and then I'm clipping for the rest of the day. (I only have 8.x kWh of battery).

What I did last year was set the charge rate to 1357 in certain conditions which meant I would always have enough inverter and charge capacity to avoid clipping, provided the battery didn't fill up. But I was on Octopus cosy last year, so export rates were lower than import rates, it was worth charging from solar.

The maths are different on Agile. I was already clipping just after 9am this morning the sun was that shiny. It's March, I wasn't expecting it. The battery was already 57%, a fully battery was possible on solar (but I didn't necessarily want that, because export rates are higher than import rates - I'd rather use the grid as a battery, export at 15p and buy back at lower prices. Winner winner chicken dinner).

So what I'm trying to create here is a solution to more dynamically manage the charge rate, to minimise solar charging when it isn't wanted, to try and avoid a full battery before the panels stop generating more than 3.6kW. The solcast forecasts are absolute garbage at the moment in terms of predicting the amount of power I'm going to generate and when, but they're good enough to base the morning battery SOC on.

Looks like it's a sunny day again tomorrow to try and do some more prototyping.

[springfall2008]

I like the idea of dynamically controlling the charge rate to reduce clipping, but it would also have to be modelled in Predbat to ensure the plan aligns to reality.

Would it not be simply the case, for most people, of setting the charge rate to array_size_kwp - inverter_size_kw rather than setting it to zero?

If so I could include that as a 3rd option between 0 and full rate charge mode?

[gcoan]

Would it not be simply the case, for most people, of setting the charge rate to array_size_kwp - inverter_size_kw rather than setting it to zero?

Not sure I follow how this would work in practice Trefor, I think there needs to be more intelligent prediction of solar clipping rather than just always charging the battery at a level that stops solar generation above the AC level of the inverter being lost.

e.g. today, very sunny first thing, my panels were generating 4.7kW of which a theoretical maximum of 2.6 was going into the battery (actually I think it was 2.4 in reality), and the rest was exported.

But by lunchtime it started raining and solar generation dropped right off to just under 1kW.

If I'd been throttling charging to 1.2kW (my excess of array vs inverter size) then I'd have ended up with a partially filled battery.

Maybe something like:

max_ac_charging_kwh = (number of hours of solar remaining * max inverter AC rate)
potential_clipped_kwh = SUM(PV prediction remaining today) - max_ac_charging_kwh

IF potential_clipped_kwh > 0
then:
# set charge rate to capture the clipped energy
  charge rate = potential_clipped_kwh / (number of hours of solar remaining)
# and add on charge rate required to hit battery target SoC
  charge rate += (target SoC in Wh - current SoC in Wh) / (number of hours of solar remaining)

else:
  charge rate = battery max charge rate

And to repeat my point of earlier, multiple arrays/inverters will make this determination of what is the appropriate charge rate harder.

[slopemad]

I like the idea of dynamically controlling the charge rate to reduce clipping, but it would also have to be modelled in Predbat to ensure the plan aligns to reality.

So Predbat could use the Solcast estimates to decide what the morning SOC should be, and use a reduced charge rate on that basis (probably based on array_size_kwp - inverter_size_kw as a maximum?) during those peaks.

Would it not be simply the case, for most people, of setting the charge rate to array_size_kwp - inverter_size_kw rather than setting it to zero?

For me, with such intermittent cloud (and then rain, and then lots of rain, followed by more rain), that would have just put too much charge in the battery today and at the height of summer, would fill the battery with power it could export directly, but I could see that being fine for people with HUGE batteries. The Solcast forecast for today for me had a peak prediction of 3.4kW (even the PV90 forecast wasn't any more than 3.5kW).

The concept I'm going to look at tomorrow is to increase the charge rate by 416W every minute until the inverter is no longer at it's maximum rate (so, in theory, maximum of 3 minutes to respond to sunshine), and only wind it back down as the 5 minute peak PV rate reduces, and see how that looks.

[slopemad]

I've been looking at this again and making tweaks to the code. Right now it's just monitoring and not actually changing anything, but if anyone else wants to have a look/test that might be useful. Some of the code is pretty crufty as it's just prototyping at the moment. I am just running this to get running data out of the logs:

This code runs every 60 seconds, unless the levels of solar are regularly changing (i.e. intermittent cloud cover) in which case it runs every 30 seconds, and at the moment just looks back through the past 4 minutes of data. It wants to increase the charge_rate as soon as practically possible, but it wont reduce charge rate until that peak has disappeared out of the 4 minutes of data, to avoid too much changing of the settings.

This, in my mind, is different to how predbat itself models its predictions which is currently fine. This script isn't predicting, it is just there to avoid clipping, and avoiding putting more solar in the battery when it's better to export as much solar as possible. (Today, I'm better off exporting as much solar as I can, and then if there's a shortfall for the evening peak, I can charge the battery as 12p later this afternoon. The only solar going in the battery is the 'free' solar which I wouldn't be able to export due to exceeding the inverter capacity).

grep PVCalc /config/appdaemon.log

This is pvcalc.yaml

PVCalc::
  module: PVCalc
  class: PVCalc

  # Sets the prefix for all created entities in HA - only change if you want to run more than once instance
  prefix: pvcalc

  # Timezone to work in
  timezone: Europe/London

This is PVCalc.py:

import appdaemon.plugins.hass.hassapi as hass
import datetime
import time
import statistics
import json
import requests

class PVCalc(hass.Hass):

    def initialize(self):
        self.rest_api = "http://homeassistant.local:6345"
        self.pvgen_entity = "sensor.givtcp_XXXXXXXXXX_pv_power"
        self.inverter_entity = "sensor.givtcp_XXXXXXXXXX_invertor_power"
        self.forecast_now_entity = "sensor.solcast_pv_forecast_power_now"
        self.forecast_next_entity = "sensor.solcast_pv_forecast_power_next_30_mins"
        self.current_charge_rate_entity = "number.givtcp_XXXXXXXXXX_battery_charge_rate"

        #Adjust as appropriate
        self.inverter_maxrate = 3600
        self.charge_maxrate = 2600
        self.solar_max_kwp = 4900

        self.inverter_loss = 0.04
        self.charge_loss = 0.067
        self.discharge_loss = 0.05
        self.charge_rate_steps = 0.04

        # Schedule the data collection function to run every 5 minutes
        self.frequency = 60
        self.collect_sensor_data_handle = None
        self.collect_sensor_data()
        self.collect_sensor_data_handle = self.run_every(self.collect_sensor_data, datetime.datetime.now() + datetime.timedelta(seconds=5), self.frequency)

    def collect_sensor_data(self, kwargs=None):
        self.log("--------------- PVCALC START RUN ---------------")
        # Calculate the start and end time for the data collection (last 5 minutes)
        end_time = datetime.datetime.now()
        start_time = end_time - datetime.timedelta(minutes=3)

        # Retrieve sensor data from Home Assistant

        pvgen_data = self.get_history_async(self.pvgen_entity, start_time, end_time)
        inverter_data = self.get_history_async(self.inverter_entity, start_time, end_time)
        pvgen_values = [int(state['state']) for states in pvgen_data for state in states]
        inverter_values = [int(state['state']) for states in inverter_data for state in states]
        forecast_now_value = self.get_state(self.forecast_now_entity)
        forecast_next_value = self.get_state(self.forecast_next_entity)
        current_charge_rate = int(self.get_state(self.current_charge_rate_entity))
        new_charge_rate = current_charge_rate

        charge_maxrate = self.charge_maxrate
        solar_max_kwp = self.solar_max_kwp
        inverter_loss = self.inverter_loss
        charge_loss = self.charge_loss
        discharge_loss = self.discharge_loss
        charge_rate_steps = self.charge_rate_steps

        pvgen_min = min(pvgen_values)
        pvgen_max = max(pvgen_values)
        pvgen_max_last = max(pvgen_values[-1], pvgen_values[-2])
        pvgen_avg = int(statistics.mean(pvgen_values))
        pvgen_last = pvgen_values[-1]
        if len(pvgen_values) >= 2:
            pvgen_stddev = int(statistics.stdev(pvgen_values))
        else:
            pvgen_stddev = None

        inverter_min = min(inverter_values)
        inverter_max = max(inverter_values)
        inverter_max_last = max(inverter_values[-1], inverter_values[-2])
        inverter_avg = int(statistics.mean(inverter_values))
        inverter_last = inverter_values[-1]
        if len(inverter_values) >= 2:
            inverter_stddev = int(statistics.stdev(inverter_values))
        else:
            inverter_stddev = None

        # Process collected sensor data (example: printing)
        self.log("PV Generation Data Collected from {} to {}: {}".format(start_time, end_time, pvgen_values))
        self.log("Inverter Data Collected from {} to {}: {}".format(start_time, end_time, inverter_values))
        self.log("PV Generation Statistics - Min: {}, Max: {}, Last: {}, Average: {}, Std Deviation: {}".format(pvgen_min, pvgen_max, pvgen_last, pvgen_avg, pvgen_stddev))
        self.log("Inverter Statistics - Min: {}, Max: {}, Last: {}, Average: {}, Std Deviation: {}".format(inverter_min, inverter_max, inverter_last, inverter_avg, inverter_stddev))

        self.log("Solcast Forecast now: {}W".format(forecast_now_value))
        self.log("Solcast Forecast next: {}W".format(forecast_next_value))
        self.log("Current charge rate: {}".format(current_charge_rate))

        expected_max_solar = int((self.inverter_maxrate / (1 - inverter_loss)) + current_charge_rate)
        self.log("Expected Max Solar: {}".format(expected_max_solar))

        if inverter_last < 0:
            # AC Charging, charge_rate wants to be what agilebat says it should be here
            self.log("Appears to be charging from AC.  No change to charge rate.")
            new_charge_rate = None
        elif inverter_last > pvgen_last:
            # Forced Discharge?  charge_rate should be what agilebat says.
            self.log("Inverter throughput exceeding PV generation.  No change to charge rate.")
            new_charge_rate = None
        else:
            if pvgen_max < self.inverter_maxrate * 0.95:
                # If PV Generation is lower than inverter max then no point diverting solar to battery
                self.log("PV Generation ({}) is below inverter max rate threshold ({}).  No point diverting solar to battery.".format(pvgen_max, self.inverter_maxrate * 0.95))
                new_charge_rate = 0
            else:
                expected_max_solar_threshold = int(expected_max_solar * 0.95)
                self.log("Expected Max Solar Threshold {}".format(expected_max_solar_threshold))
                #If the max being generated sits at around the capacity to deal with it (i.e. inverter + charger) then increase the charge rate
                if abs(pvgen_max - expected_max_solar_threshold) < 104:
                    #Don't bother if the difference is minimal
                    self.log("Generation ({}) close to threshold ({}), no point adjusting".format(pvgen_max, expected_max_solar_threshold))  
                elif (pvgen_max > expected_max_solar_threshold):
                    #Does charge rate need to be increased?
                    self.log("PV Generation ({}) is near maximum expected ({}).  Increase charge rate.".format(pvgen_max, expected_max_solar_threshold))
                    #new_charge_rate = current_charge_rate + (charge_maxrate * (charge_rate_steps * 2)) + 5
                    new_charge_rate = min(current_charge_rate + (pvgen_stddev + (charge_maxrate * charge_rate_steps) + 2), 1254)
                else:
                    #Does charge rate need to be reduced?
                    self.log("PV Generation ({}) is below expected max solar threshold ({}).  Reduce charge rate.".format(pvgen_max, expected_max_solar_threshold))
                    #new_charge_rate = current_charge_rate
                    new_charge_rate = pvgen_max - self.inverter_maxrate + (charge_maxrate * charge_rate_steps)

        if new_charge_rate is not None:
            new_charge_rate = int(min(int(new_charge_rate), 1253))

        self.log("Proposed new charge rate: {}".format(new_charge_rate))

        if new_charge_rate is not None and abs(new_charge_rate - current_charge_rate) > 104:
            # Uncomment this to make it actually change the charge rate
            #self.set_charge_rate(new_charge_rate)
        else:
            self.log("Not changing charge rate")

        #Change running frequency if it is cloudy (high generation std dev)
        if self.collect_sensor_data_handle is not None:
            if (pvgen_stddev > 120):
                if self.frequency == 60:
                    self.frequency = 30
                    self.reschedule_task()
            else:
                if self.frequency == 30:
                    self.frequency = 60
                    self.reschedule_task()

        self.log("--------------- PVCALC END RUN ---------------")

    def reschedule_task(self):
        self.cancel_timer(self.collect_sensor_data_handle)
        self.collect_sensor_data_handle = self.run_every(self.collect_sensor_data, datetime.datetime.now() + datetime.timedelta(seconds=5), self.frequency)

    async def get_history_async_hook(self, result, entity_id, start_time, end_time):
        """
        Async function to get history from HA
        """
        if start_time and end_time:
            result["data"] = await self.get_history(entity_id=entity_id, start_time=start_time, end_time=end_time)
        else:
            result["data"] = await self.get_history(entity_id=entity_id)

    def get_history_async(self, entity_id, start_time=None, end_time=None):
        """
        Async function to get history from HA using Async task
        """
        result = {}
        task = self.create_task(self.get_history_async_hook(result, entity_id=entity_id, start_time=start_time, end_time=end_time))
        cnt = 0
        while not task.done() and (cnt < 120):
            time.sleep(0.05)
            cnt += 0.05

        if "data" in result:
            return result["data"]
        else:
            self.log("Failure to fetch history for {}".format(entity_id))
            raise ValueError

    def set_charge_rate(self, charge_rate=2600):
        api = "setChargeRate"
        url = self.rest_api + "/" + api
        # charge_rate expressed as a percentage of the maximum. (an integer from 0-100)
    # Apparently this isn't true,

        if not (0 <= charge_rate <= 2600):
            self.log("Failed.  Charge Rate must be between 0 and 2600")
            return None

        payload = {
            "chargeRate": charge_rate
        }
        api_payload = json.dumps(payload)

        self.log("Setting Charge Rate to {}".format(charge_rate))
        try:
            response = requests.post(url, data=api_payload)
            response.raise_for_status()
            self.log("Charge Rate successfully set")
        except requests.exceptions.RequestException as e:
            self.log("Error setting Charge Rate {}".format(e))

[gcoan]

@slopemad I've tried installing it. Copying the pvcalc.yaml and PVCalc.py into /appdaemon/apps. Took several times to get it to load because I'd mis-named the .py as PVcalc.py

I customised the givtcp sensor names, increased the inverter max rate and solar charge kwp.

Anyway, loaded, gave the initial started log then nothing more in the appdaemon.log file that predbat was busy writing to

After a while of waiting for it, I found in the appdaemon add-on Log the following error crash:

2024-03-30 13:00:38.003562 WARNING PVCalc:: ------------------------------------------------------------
2024-03-30 13:00:38.004135 WARNING PVCalc:: Unexpected error running initialize() for PVCalc:
2024-03-30 13:00:38.005442 WARNING PVCalc:: ------------------------------------------------------------
2024-03-30 13:00:38.008490 WARNING PVCalc:: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
    await utils.run_in_executor(self, init)
  File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
    response = future.result()
               ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/homeassistant/appdaemon/apps/PVCalc.py", line 29, in initialize
    self.collect_sensor_data()
  File "/homeassistant/appdaemon/apps/PVCalc.py", line 42, in collect_sensor_data
    pvgen_values = [int(state['state']) for states in pvgen_data for state in states]
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/homeassistant/appdaemon/apps/PVCalc.py", line 42, in <listcomp>
    pvgen_values = [int(state['state']) for states in pvgen_data for state in states]
                    ^^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: '0.005'

2024-03-30 13:00:38.009229 WARNING PVCalc:: ------------------------------------------------------------

Not sure what it is rejecting

[slopemad]

It looks like the state of the pvgen_entity when being pulled from the history is a floating point number rather than something the code can turn into an integer.

After this line: pvgen_data = self.get_history_async(self.pvgen_entity, start_time, end_time)

Can you add self.log("PVGEN DATA: {}".format(pvgen_data))

i.e.

        pvgen_data = self.get_history_async(self.pvgen_entity, start_time, end_time)
        self.log("PVGEN DATA: {}".format(pvgen_data))

Then you should at least get a "PVGEN DATA" line in appdaemon.log?

[gcoan]

It is indeed a float coming back:

2024-03-30 13:25:52.298833 INFO AppDaemon: Terminating PVCalc:
2024-03-30 13:25:52.302414 INFO AppDaemon: Reloading Module: /homeassistant/appdaemon/apps/PVCalc.py
2024-03-30 13:25:52.306253 INFO AppDaemon: Loading app PVCalc: using class PVCalc from module PVCalc
2024-03-30 13:25:52.310817 INFO AppDaemon: Calling initialize() for PVCalc:
2024-03-30 13:25:52.313573 INFO PVCalc:: --------------- PVCALC START RUN ---------------
2024-03-30 13:25:52.392781 INFO PVCalc:: PVGEN DATA: [[{'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.005', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T13:21:52+00:00', 'last_updated': '2024-03-30T13:21:52+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.559', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T13:25:46.708403+00:00', 'last_updated': '2024-03-30T13:25:46.708403+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}]]

[gcoan]

Ah, I know, I have the UoM set to kW not W

[slopemad]

Yep, you'll definitely want unit of measurement to be W rather than kW. (When I pull the history, I get mine back in Watts). The script is working in tiny units, like 100W, so it'll struggle to worth with data in kW because 0.005kW = 5000W, just isn't granular enough for this purpose.

I'm not sure where the uom as kW came from?

[slopemad]

Spot the massive error ;-). 0.005kW is 5W ;-). And indeed 'state': '0.559' would be 559W.

This might work if your uom is kW...

pvgen_values = [int(float(state['state']) * 1000) for states in pvgen_data for state in states]

[gcoan]

You can change the uom of any power based entities between W and kW. I have all mine changed to kW so its easier to read on dashboards, I'm more interested in knowing its generating 2.4kW than 2319W.

It just needs some logic adding to look at the UoM and convert it to Watts (x1000) if that's what you need to deal with. Sloppy programming if you ask me.....

[slopemad]

Definite sloppy programming, I hadn't considered uom might not be watts. I'll make the code in this area a bit more robust.

[gcoan]

I have power graphs like this which is why its all kW:

[gcoan]

I've changed the pvgen_values line as suggested.

Its crashing again a bit further on

2024-03-30 13:43:25.081121 WARNING PVCalc:: ------------------------------------------------------------
2024-03-30 13:43:26.355875 WARNING PVCalc:: ------------------------------------------------------------
2024-03-30 13:43:26.356675 WARNING PVCalc:: Unexpected error running initialize() for PVCalc:
2024-03-30 13:43:26.357041 WARNING PVCalc:: ------------------------------------------------------------
2024-03-30 13:43:26.360055 WARNING PVCalc:: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
    await utils.run_in_executor(self, init)
  File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
    response = future.result()
               ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/homeassistant/appdaemon/apps/PVCalc.py", line 29, in initialize
    self.collect_sensor_data()
  File "/homeassistant/appdaemon/apps/PVCalc.py", line 69, in collect_sensor_data
    inverter_min = min(inverter_values)
                   ^^^^^^^^^^^^^^^^^^^^
ValueError: min() arg is an empty sequence

2024-03-30 13:43:26.360755 WARNING PVCalc:: ------------------------------------------------------------

[slopemad]

I think this will be the fix in the code as appropriate:

pvgen_values = []
for states in pvgen_data:
    for state in states:
        value = state['state']
        if state['attributes']['unit_of_measurement'] == 'kW':
            value = int(float(value) * 1000)
        else:
            value = int(value)
        pvgen_values.append(value)

And, of course, at this prototyping stage, it is based around my setup, i.e. controlling a single hybrid inverter. I guess it needs to be very different logic when you have seperate string inverters and two different AC coupled chargers.

[gcoan]

Adding that code in with extra debug lines shows it is now working. I thought we would have the same issue with inverter data, but its not something I have changed uom on so its working OK (but would need changing for resilience)

2024-03-30 14:42:42.369535 INFO PVCalc:: --------------- PVCALC START RUN ---------------
2024-03-30 14:42:42.439529 INFO PVCalc:: PVGEN DATA b4: [[{'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.367', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:38:42+00:00', 'last_updated': '2024-03-30T14:38:42+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.357', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:39:06.761443+00:00', 'last_updated': '2024-03-30T14:39:06.761443+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.350', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:39:32.788947+00:00', 'last_updated': '2024-03-30T14:39:32.788947+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.342', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:40:15.767755+00:00', 'last_updated': '2024-03-30T14:40:15.767755+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.343', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:40:26.703140+00:00', 'last_updated': '2024-03-30T14:40:26.703140+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.404', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:41:29.642840+00:00', 'last_updated': '2024-03-30T14:41:29.642840+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.417', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:41:56.023090+00:00', 'last_updated': '2024-03-30T14:41:56.023090+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.425', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:42:22.433738+00:00', 'last_updated': '2024-03-30T14:42:22.433738+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}]]
2024-03-30 14:42:42.442766 INFO PVCalc:: PVGEN DATA after: [367, 357, 350, 342, 343, 404, 417, 425]
2024-03-30 14:42:42.503907 INFO PVCalc:: inverter DATA b4: [[{'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '374', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:38:42+00:00', 'last_updated': '2024-03-30T14:38:42+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '366', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:39:06.775309+00:00', 'last_updated': '2024-03-30T14:39:06.775309+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '359', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:39:32.802418+00:00', 'last_updated': '2024-03-30T14:39:32.802418+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '352', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:40:15.813950+00:00', 'last_updated': '2024-03-30T14:40:15.813950+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '406', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:41:29.662628+00:00', 'last_updated': '2024-03-30T14:41:29.662628+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '420', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:41:56.067631+00:00', 'last_updated': '2024-03-30T14:41:56.067631+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '429', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:42:22.435282+00:00', 'last_updated': '2024-03-30T14:42:22.435282+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}]]
2024-03-30 14:42:42.508305 INFO PVCalc:: PV Generation Data Collected from 2024-03-30 14:38:42.369821 to 2024-03-30 14:42:42.369821: [367, 357, 350, 342, 343, 404, 417, 425]
2024-03-30 14:42:42.509221 INFO PVCalc:: Inverter Data Collected from 2024-03-30 14:38:42.369821 to 2024-03-30 14:42:42.369821: [374, 366, 359, 352, 406, 420, 429]
2024-03-30 14:42:42.510084 INFO PVCalc:: PV Generation Statistics - Min: 342, Max: 425, Last: 425, Average: 375, Std Deviation: 34
2024-03-30 14:42:42.510968 INFO PVCalc:: Inverter Statistics - Min: 352, Max: 429, Last: 429, Average: 386, Std Deviation: 31
2024-03-30 14:42:42.511850 INFO PVCalc:: Solcast Forecast now: NoneW
2024-03-30 14:42:42.512721 INFO PVCalc:: Solcast Forecast next: NoneW
2024-03-30 14:42:42.513606 INFO PVCalc:: Current charge rate: 2600
2024-03-30 14:42:42.514512 INFO PVCalc:: Expected Max Solar: 7808
2024-03-30 14:42:42.517816 INFO PVCalc:: Inverter throughput exceeding PV generation.  No change to charge rate.
2024-03-30 14:42:42.519270 INFO PVCalc:: Proposed new charge rate: None
2024-03-30 14:42:42.522081 INFO PVCalc:: --------------- PVCALC END RUN ---------------

Suspicious that the forecast now and next is empty: 2024-03-30 14:49:36.702871 INFO PVCalc:: forecast_now_value: None 2024-03-30 14:49:36.704929 INFO PVCalc:: forecast_next_value: None 2024-03-30 14:49:36.706164 INFO PVCalc:: current_charge_rate: 2600

[gcoan]

Fixed it, I had renamed my solcast entities as they'd come out something like solcast_solcast_forecast and I'd renamed them to solcast

2024-03-30 14:52:02.114926 INFO PVCalc:: --------------- PVCALC START RUN ---------------
2024-03-30 14:52:02.168593 INFO PVCalc:: PVGEN DATA b4: [[{'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.422', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:48:02+00:00', 'last_updated': '2024-03-30T14:48:02+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.380', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:48:28.033238+00:00', 'last_updated': '2024-03-30T14:48:28.033238+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.416', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:48:54.173035+00:00', 'last_updated': '2024-03-30T14:48:54.173035+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.423', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:49:20.399496+00:00', 'last_updated': '2024-03-30T14:49:20.399496+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.424', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:49:46.572577+00:00', 'last_updated': '2024-03-30T14:49:46.572577+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.005', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:50:38.722514+00:00', 'last_updated': '2024-03-30T14:50:38.722514+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.097', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:51:04.798617+00:00', 'last_updated': '2024-03-30T14:51:04.798617+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.005', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:51:30.845724+00:00', 'last_updated': '2024-03-30T14:51:30.845724+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_pv_power', 'state': '0.004', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'kW', 'device_class': 'power', 'icon': 'mdi:solar-power', 'friendly_name': 'G SD2237G182 Power G PV Power'}, 'last_changed': '2024-03-30T14:51:56.885465+00:00', 'last_updated': '2024-03-30T14:51:56.885465+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}]]
2024-03-30 14:52:02.170657 INFO PVCalc:: PVGEN DATA after: [422, 380, 416, 423, 424, 5, 97, 5, 4]
2024-03-30 14:52:02.225636 INFO PVCalc:: inverter DATA b4: [[{'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '429', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:48:02+00:00', 'last_updated': '2024-03-30T14:48:02+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '390', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:48:28.059470+00:00', 'last_updated': '2024-03-30T14:48:28.059470+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '422', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:48:54.223609+00:00', 'last_updated': '2024-03-30T14:48:54.223609+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '432', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:49:20.404302+00:00', 'last_updated': '2024-03-30T14:49:20.404302+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '434', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:50:12.736460+00:00', 'last_updated': '2024-03-30T14:50:12.736460+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '0', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:50:38.760552+00:00', 'last_updated': '2024-03-30T14:50:38.760552+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '244', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:51:04.808777+00:00', 'last_updated': '2024-03-30T14:51:04.808777+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}, {'entity_id': 'sensor.g_sd2237g182_invertor_power', 'state': '0', 'attributes': {'state_class': 'measurement', 'unit_of_measurement': 'W', 'device_class': 'power', 'friendly_name': 'G SD2237G182 Power G Invertor Power'}, 'last_changed': '2024-03-30T14:51:30.847296+00:00', 'last_updated': '2024-03-30T14:51:30.847296+00:00', 'context': {'id': '01HT54NX2MQD58P2P1KHNK5EBW', 'parent_id': None, 'user_id': None}}]]
2024-03-30 14:52:02.228312 INFO PVCalc:: forecast_now_value: 1812
2024-03-30 14:52:02.234845 INFO PVCalc:: forecast_next_value: 1826
2024-03-30 14:52:02.236282 INFO PVCalc:: current_charge_rate: 2600
2024-03-30 14:52:02.238347 INFO PVCalc:: PV Generation Data Collected from 2024-03-30 14:48:02.115661 to 2024-03-30 14:52:02.115661: [422, 380, 416, 423, 424, 5, 97, 5, 4]
2024-03-30 14:52:02.240825 INFO PVCalc:: Inverter Data Collected from 2024-03-30 14:48:02.115661 to 2024-03-30 14:52:02.115661: [429, 390, 422, 432, 434, 0, 244, 0]
2024-03-30 14:52:02.242214 INFO PVCalc:: PV Generation Statistics - Min: 4, Max: 424, Last: 4, Average: 241, Std Deviation: 205
2024-03-30 14:52:02.244030 INFO PVCalc:: Inverter Statistics - Min: 0, Max: 434, Last: 0, Average: 293, Std Deviation: 191
2024-03-30 14:52:02.245657 INFO PVCalc:: Solcast Forecast now: 1812W
2024-03-30 14:52:02.246996 INFO PVCalc:: Solcast Forecast next: 1826W
2024-03-30 14:52:02.248216 INFO PVCalc:: Current charge rate: 2600
2024-03-30 14:52:02.249778 INFO PVCalc:: Expected Max Solar: 7808
2024-03-30 14:52:02.253318 INFO PVCalc:: PV Generation (424) is below inverter max rate threshold (4750.0).  No point diverting solar to battery.
2024-03-30 14:52:02.255383 INFO PVCalc:: Proposed new charge rate: 0
2024-03-30 14:52:02.257651 INFO PVCalc:: --------------- PVCALC END RUN ---------------

[gcoan]

The Predbat solcast figure on my plan for this slot is 0.15kWh. Currently off the givenergy two arrays that are configured in solcast I'm getting 0.5kW and 1.8kW, so very much above forecast level!

(I have a 3rd FIT array which is generating a further 2.9kW right now but as this is AC only and not available to my hybrid inverters, I don't configure this in solcast so its "bunce" for predbat)

[slopemad]

For me now, on my South facing array, I'm now beyond the point of the day when I'm generating more than my inverter can handle. And whilst it's been cloudy for much of the afternoon, it's now quite clear. So whilst I could be charging from grid at 12p, I can't because there's too much solar (which I can export for 15p). And it's touch and go whether there will be a cheap enough slot this evening to be worth charging from the grid at, when you account for inverter losses.

(1kWh of solar = 960W exported, of course. And I'd need to import 1040W to make up for the 960W exported when it comes to battery charging).

[gcoan]

Fortunately one bit my installers got right was using 5kW inverters so at the moment I'm not yet getting a lot of clipping on the G inverter which is 16 panels on the East front of the roof, and on H which is 6 East facing and 6 West I'm well below clipping levels.

But it'll only increase as summer comes on

[gcoan]

Sunny day today and looks like my front array was peaking out around 4.8kW so thought I'd see what pvcalc was saying but found it wasn't running.

Had crashed when I restarted HA a day ago:

2024-03-31 02:17:06.207324 WARNING PVCalc:: ------------------------------------------------------------
2024-03-31 02:17:06.207663 WARNING PVCalc:: Unexpected error running initialize() for PVCalc:
2024-03-31 02:17:06.207836 WARNING PVCalc:: ------------------------------------------------------------
2024-03-31 02:17:06.212247 WARNING PVCalc:: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
    await utils.run_in_executor(self, init)
  File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
    response = future.result()
               ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/homeassistant/appdaemon/apps/PVCalc.py", line 29, in initialize
    self.collect_sensor_data()
  File "/homeassistant/appdaemon/apps/PVCalc.py", line 50, in collect_sensor_data
    value = int(float(value) * 1000)
                ^^^^^^^^^^^^
ValueError: could not convert string to float: 'unavailable'

2024-03-31 02:17:06.212393 WARNING PVCalc:: ------------------------------------------------------------

I'll restart appdaemon, should get it going again

[Neomancer86]

I'll also have a go at deploying and testing this.

I've been using the slow charging feature in predbat as kinda a stopgap for this feature till I discovered it today!

[gcoan]

@slopemad just wondering whatever happened to this and whether you got anywhere towards incorporating this into a PR for predbat? I see there are other recent tickets #1206 and #1152 about clipping and what can be done about it in predbat. Definitely something that needs addressing for many users

[PatrickJanssens]

My two cents worth on this issue. I.m relatively new to home assistant, GivTCP and predbat, so haven't figured out how to properly automate this. but I think the key is to ensure the battery SOC is sufficiently low prior to the time clipping is expected. I target to get it to 10% on a very sunny day, perhaps closer to 50% if intermittently cloudy. I use Solcast forecast to estimate how much excess energy from the excess PV DC I may have to store. (Solcast date actually gives me "clipped" estimates as it takes account of the inverter capacity, so I estimate the excess based on the how long time it predicts PV power will exceed clipping power). As soon as clipping is happening I set an automation to switch off the Eco mode and set the system to Timed Discharge, still with the above mentioned discharge SOC% (this can be done either based on the Solcast forecast, or I triggered it by the first time the actual PV power reaches the clipping point). I find this way the excess PV power is sent to the battery, but when PV production drops below the inverter capacity the battery is discharging again to increase the buffer for more excess PV. I don't think there is a need to adjust the battery charge or discharge rates for this. As long as it can be configured to only send the excess PV to the battery (seems to work fine on my GivEnergy hybrid inverter using timed discharge). The limiting factor is really the battery capacity, so key is to start with an as low as possible battery SOC, and ensure it can discharge to the grid when no clipping is occuring to make space for more excess PV. In order to ensure I have a full battery when the sun sets I could gradually increase the discharge SOC%. But for now I just set my system to revert back to eco mode and disable the discharge schedule once the remaining PV foreceast energy drops close to how much charge I need for the battery, plus estimated consumption and some margin. (I presently set this threshhold arbitrarily to 12 to 15 kWh). This way I'm sure to have a full battery when the sun sets. but I'm sure PredBat could determine this time more accurately.

[Abn8mTGsijbJ]

Two more cents from a slightly different perspective....

With my Gen 3 hybrid inverter and my charge power set to 0 the battery still charges at about 300W from solar.

When Eco mode is turned on and the battery SoC is 100% I see a constant discharge power of approx. 60W.

I haven't gotten around to mapping out the logic yet but I'm planning to try a "simple" automation that has no reliance on load prediction or solar forecast:

1) Ensure the battery is below a certain SoC threshold prior to sunrise (to ensure room for clipped energy)

2) Between sunrise and sunset switch off eco mode (to maximise export) unless the battery SoC is below a certain threshold (e.g. due to no cheap overnight import slots). This is because there is usually an overnight slot cheap enough to make it more cost effective to charge from the grid rather than solar.

3) Constantly (every 20s) check the battery SoC and when it reaches the desired minimum threshold turn off eco mode (to maximise export)

4) Providing the import price is above a certain threshold, constantly (every 20s) compare GivTCP load power to GivTCP PV power and whenever demand is above solar generation switch eco mode back on and then switch it off again when the load falls below PV power (this is the same intent as the person above to account for intermittent clouds or a high load appliance being switched on)

5) Between sunset and sunrise switch on eco mode (to minimise import) unless the import price is below a certain threshold

6) Find the cheapest charge slots overnight and if there below a certain threshold charge the battery (otherwise let the pv do it the following morning) My battery capacity is 9.5kWh and my daily usage is about the same so I can make it through about 3 days with either very little solar or no cheap overnight slots.

I'd need to find a way to slip in a forced export down to 4% every few days somewhere between sunset and sunrise to allow the battery management system to re-calibrate the SoC currently I do this manually when I see a few cheap slots

GivTCP is already polling the inverter every 20s so it shouldn't overload it and the longest I should be importing or exporting when I don't want to would be around 40s (between eco mode switches)

[gcoan]

With my Gen 3 hybrid inverter and my charge power set to 0 the battery still charges at about 300W from solar.

When Eco mode is turned on and the battery SoC is 100% I see a constant discharge power of approx. 60W.

If you use battery_pause_mode and set this to Charge Disabled it drops the power consumption quite a bit.

What you described above should work. I would caution that there is a finite time for the inverter to swap mode and start charging or discharging (there's physical relays inside that you can hear clicking on or off). Personally I wouldn't try instructing the inverter every 20 seconds. I'd do it every 5 minutes or so to reduce physical wear and tear and let the inverter catch up.

I'm leaving my inverter on a low (500-900W) charge rate whilst agile rates are bad overnight, to charge up slowly in the day. But if I had a cheaper overnight rate I would turn solar charging off entirely.

[BuhJuhWuh]

Forgive me if I'm missing something obvious here, but shouldn't it be fairly straightforward to set the effective inverter limit based on the battery SoC?

So in its simplest form (using my system as an example, nominal inverter limit = 5kW, max DC input = 7.5kW):

• if SoC >90%, inverter limit = 5kW
• else, inverter limit = 7.5kW

Then wouldn't Predbat's existing cost optimisation logic do the heavy lifting, prioritising max export (and trickle charging the battery with any excess once load is taken care of) until the last possible forecast moment when it can hit full charge before the 4pm Agile peak?

If the hard threshold proves unstable, a tapering lookup table implemented in exactly the same way as the charging curve could do the trick.

[gcoan]

So in its simplest form (using my system as an example, nominal inverter limit = 5kW, max DC input = 7.5kW):

if SoC >90%, inverter limit = 5kW else, inverter limit = 7.5kW

I would say no, it needs to be more sophisticated than this.

On a sunny day my batteries will fully charge by 10am if set to the default max charging rate. At which point any generation above the max AC output rate will be clipped and lost.

To maximise generation and export we want to trickle charge the battery as much as possible so that it doesn't fill up too early. At the moment for example Predbat is in read only mode and the battery charge rate is set to 770W so it will take most of the day to charge up but I'll be able to export 5kW during that period alongside the battery charging.

Ideally the charge rate would be determined by predicted solar and predbat would use a combination of pause charge early in the day (when solar generation < inverter AC limit) and low charge rate (when solar generation > inverter AC limit) to maximise export.

[BuhJuhWuh]

I agree with you on the desired behaviour, but wouldn't Predbat's existing optimisation logic result in something very close to this, because plans which keep SoC below the threshold until the last minute (based on the available load history and solar forecast already being used) would win out on cost because of the extra generation?

Or is the problem that Predbat does not have a mode where it can prioritise Load > Export > Charge (before switching to Load > Charge > Export later)?

[gcoan]

plans which keep SoC below the threshold until the last minute (based on the available load history and solar forecast already being used)

the existing predbat logic would definitely help, but I think the missing piece is the ability to dynamically work out what the charge rate should be. At the moment this is either retrieved from the inverter or over-ridden in apps.yaml, but it's effectively a simple on/off switch to charge the battery.

Taking my own Gen 1 hybrid as an example, from the data sheet https://kb.givenergy.cloud/article.php?id=101&oid=19 the max DC output is 6500W, the max AC is 5000W and the max battery charge is 2600W.

Predbat will at present only turn on/turn off battery charging at 2.6kW.

Just trying to follow the maths through:

Assuming steady 6.5kW generation for 2 hours.

Strategy 1, charge battery for an hour, stop charging for an hour. 2.6kW fed into the battery 6.5-2.6=3.9kW exported in hour 1, 5kW (clipped output) in hour 2 = 8.9kW total export

Strategy 2, charge battery at 50% charge rate for 2 hours 2.6kW fed into the battery (1.3kW for each hour) 5kW exported for both hour 1 and hour 2 = 10kW

The addition of more intelligence to the charge rate is crucial

[PatrickJanssens]

I really don’t see the benefit of trying to control the charge rate or discharge rate. My comment is based on my setup, which is a GivEnergy hybrid inverter (5 kW gen 3), GivEnergy 9.5 kWh battery and a 7.83 kWp solar array. I’m on the Intelligent Octopus Go tariff, so I do want to charge my battery to 100% prior 5:30 am. From what I have been trying out the key is to ensure the battery charge is as low as possible prior clipping is starting. The difficulty is to calculate how much time the battery will need to discharge to that level, as it is limited by how much solar is produced at the same time (solar+battery discharge can’t exceed the inverter AC capacity). I would think this is something that predbat should be good at doing. Once the clipping point is reached configuring the battery as timed discharge (eco mode off) does exactly what you need it to do: convert excess DC from the solar panels to charge the battery whilst exporting to the grid at the maximum inverter capacity. If at any point the solar power is dropping below the clipping point the battery will discharge to the grid at maximum capacity, making more battery capacity available when clipping resumes. For all of these you really want to maintain the maximum available charge and discharge rates. As we finally have some decent sunshine here in the UK I tried following today. I configured predbat to start a Manual Forced Discharge starting from 7:00 up to 16:00. As the morning was rather cloudy the battery reached 10% capacity by 9:15 already, and stayed at that level. (10% is the SOC level set in GivEnergy Cloud Remote Control – seems predbat is not controlling this?). The first clipping PV level was reached at around 10:15. The only issue I had is that predbat(?) meanwhile changed the battery charge rate of the inverter to 0 W. So had to manually change that back to full power to resume the battery charging using surplus DC. Other than that my inverter behaved as expected, maximising the use of the available power over the clipping point. The next key point is to timely revert back to the Eco mode, so that the battery is sufficiently charged for the evening load. As mentioned earlier I do this based on the Solcast PV remaining forecast. Again I believe this is where predbat could provide a more accurate timing.

[BuhJuhWuh]

Just trying to follow the maths through:

Interesting, thanks for the explanation. I think you're right, but that's not to say there isn't also benefit to the basic "wait then charge at full rate" version.

Did a spot of very basic/hurried/unrefined spreadsheet modelling for a couple of different scenarios, with results as shown below. I've used a 7.5kWh max rate, a 5kW inverter limit and a 9.5kWh battery. It steps through a 6 hour period of 7.5kW (optimistic for my setup even this close to the solstice, but works as an illustration), in 1 hour steps.

On the left, charging starts at t=0, and the storage and export results are shown for increasing charge rates. On the right, generation rate is fixed at my system's maximum 3.3kW, but the start time is held back by the number of hours on the x-axis.

So we can gain a decent amount by just waiting, but as you say, the best results are achieved by optimising the charge rate. A combination of the two may be even better.

So yes, implementing the ability for the charge rate to be a variable, plus modelling the clipping threshold (even as a simple switch) would probably crack it, once Predbat is given free reign to do its thing!

(Itching to model it properly in Matlab now...)

[gcoan]

I really don’t see the benefit of trying to control the charge rate or discharge rate. My comment is based on my setup, which is a GivEnergy hybrid inverter (5 kW gen 3), GivEnergy 9.5 kWh battery and a 7.83 kWp solar array. I’m on the Intelligent Octopus Go tariff, so I do want to charge my battery to 100% prior 5:30 am.

Yes that makes sense, charge up at 7.5p overnight and export in the day at 15p, no point in solar charging up your battery.

Once the clipping point is reached configuring the battery as timed discharge (eco mode off) does exactly what you need it to do: convert excess DC from the solar panels to charge the battery whilst exporting to the grid at the maximum inverter capacity.

Running the numbers, there are some financial benefits of this strategy but it's not that big.

Discharging and recharging the battery incurs round trip losses of around 10-20% so you'd need to be gaining that much from capturing what would otherwise be clipped solar generation to make this worthwhile.

Interesting

I'm on Agile and at the moment overnight rates are around 15p, the same as the export rate. So I let the battery discharge overnight but slow the charge rate during the day so that it doesn't fill too quickly. Different strategies for different people

[BuhJuhWuh]

@gcoan next question: can Predbat change the charge rate (from solar) at all, or can it only affect the rate of charging from the grid?

From what I can see morning, setting predbat_battery_rate_max_scaling affects Predbat's modelling, but doesn't affect the inverter's behaviour (i.e. I set it to 0.2 but the inverter is still putting the morning's full amount of excess PV straight into the battery, unless I also change the charge power in the GE app). Is this the expected behaviour at present?

EDIT: forget that - overriding on the GE app lasted only until Predbat's next update, 5mins later... In that case, how do I tell Predbat to use a (fixed) lower rate (and is that even possible?) or delay starting to charge?

[Philhews]

I can see there are a few clipping related bugs raised on here so not sure where this is most relevant but after a quick test today, is it not possible to do the required behaviour (export first, then excess to battery) by setting 'Enable Discharge Schedule' to true and 'GivTCP Mode' to Timed Export? The only thing I haven't been able to test with this configuration (as I'm not at home) is that it still meets home demand correctly if/when the solar drops briefly. As it stands you also need to put predbat into read only mode as it will revert the above mentioned flags otherwise.

Setting charge pauses or reduced charge rates can/will still cause clipping if left in effect when the solar increases beyond AC limit (due to clouds for example) as the charge rate that has been applied will still be adhered to.

[gcoan]

is it not possible to do the required behaviour (export first, then excess to battery) by setting 'Enable Discharge Schedule' to true and 'GivTCP Mode' to Timed Export?

Unfortunately I don't think this would work with the inverter.

GivTCP mode of Timed Export is how Predbat discharges the battery to the grid. You set the start and end times you want to discharge at, the discharge rate, the battery reserve level (to % SoC discharge down to) and then when you set the mode to Timed Export it automatically flips Enable Discharge Schedule to on.

The problem is that there is no intelligence or dynamic control to the discharge/export based on solar generation, it just discharges at the set rate.

[I have scripts to force discharge so I know this is how it works]

There is another control with the newer inverter firmware, select.givtcp_serialno_local_control_mode which can be set to Load, Battery or Grid. I don't know what these do and whether this is useful or not, mine is set to Load all the time.

[Philhews]

[I have scripts to force discharge so I know this is how it works]

That's interesting because I have it currently set to Timed Export and Enable Discharge Schedule to true with prebat on read only (to stop it changing it all back to Eco etc) and it's only pushing excess solar out to the grid, no forced battery discharge. (Gen 2 hybrid if that helps). Still waiting on the load to go over the solar to confirm the battery will discharge to meet house need at that point.

The local control mode stuff appears to have precisely zero impact on my inverter so who knows what the point of that is.

[gcoan]

That's interesting because I have it currently set to Timed Export and Enable Discharge Schedule to true with prebat on read only (to stop it changing it all back to Eco etc) and it's only pushing excess solar out to the grid, no forced battery discharge. (Gen 2 hybrid if that helps). Still waiting on the load to go over the solar to confirm the battery will discharge to meet house need at that point.

Interesting behaviour.

It might be doing this because it's outside of the timed discharge period. What are discharge_start_time_slot_1 and end_time_slot_1 set to?

If the time is within these periods then it will discharge, maybe outside it acts like a hold charge?

(Mine is gen 1 hybrid but with the fast performance firmware so it should behave like a gen 2 or 3)

[Philhews]

00:00:00 for both start and end

[Philhews]

If I change the times so that I am within them then it will start dumping the battery to meet the discharge rate.

[gcoan]

@gcoan next question: can Predbat change the charge rate (from solar) at all, or can it only affect the rate of charging from the grid?

EDIT: forget that - overriding on the GE app lasted only until Predbat's next update, 5mins later... In that case, how do I tell Predbat to use a (fixed) lower rate (and is that even possible?)

At the moment Predbat by default does all charging and discharging at the max inverter rate and it gets those from requesting them from the inverter. The only control option I am aware of is to set inverter_limit_charge & _discharge in apps.yaml https://springfall2008.github.io/batpred/apps-yaml/#inverter-control-configurations

Which you can set to a lower limit to restrict the battery charge/discharge rate.

At the moment though this has to be a fixed number in apps.yaml, you can't set it to a helper entity that you could dynamically controlled in Home Assistant. I tried this as part of #1031 but it doesn't work

how do I tell Predbat to ... delay starting to charge?

One option is to manually set the select.predbat_manual_discharge_freeze to the 30 minute slots you want to prevent charging in - discharging is allowed but the SoC is otherwise frozen at the current level.

You could have an automation that sets a series of discharge freeze slots for the next day at say 11pm every night. I used to do something similar when Predbat kept on wanting to force discharge every evening just before the peak, my automation overrode the default plan

[Neomancer86]

Set Charge Low Power Mode might be helpful here.... i'm not a code wrangler so not able to help much here

switch. predbat_set_charge_low_power Enables low power charging mode where the max charge rate will be automatically determined by Predbat to be the lowest possible rate to meet the charge target. This is only really effective for charge windows >30 minutes.

I've got it enabled, it does charge slightly slower in the mornings (but still unfortunately tends to be fully charged before my peak)

[BuhJuhWuh]

At the moment Predbat by default does all charging and discharging at the max inverter rate and it gets those from requesting them from the inverter. The only control option I am aware of is to set inverter_limit_charge & _discharge in apps.yaml https://springfall2008.github.io/batpred/apps-yaml/#inverter-control-configurations

Okay, thanks, makes sense. So it does seem then that, as you say, the only way to solve this one properly is for those input parameters to become variables. That would add a nice bit of flexibility, but I can see it might be a fair bit of work to implement.

Set Charge Low Power Mode might be helpful here

Ah, interesting, thank you - hadn't clocked that setting. Will give it a whirl... well, once the next couple of cloudy-forecast days are out of the way...

[Philhews]

That's interesting because I have it currently set to Timed Export and Enable Discharge Schedule to true with prebat on read only (to stop it changing it all back to Eco etc) and it's only pushing excess solar out to the grid, no forced battery discharge. (Gen 2 hybrid if that helps). Still waiting on the load to go over the solar to confirm the battery will discharge to meet house need at that point.

Interesting behaviour.

It might be doing this because it's outside of the timed discharge period. What are discharge_start_time_slot_1 and end_time_slot_1 set to?

If the time is within these periods then it will discharge, maybe outside it acts like a hold charge?

(Mine is gen 1 hybrid but with the fast performance firmware so it should behave like a gen 2 or 3)

Well setting to timed export fell down when it needed the battery to supply the house because it didn't. I'm now trying timed demand which seems to be working for the house supply but I'll have to wait to morning to see if it still prevents charging.

[Philhews]

Well this is useless, Timed Demand doesn't prevent solar charging, so is basically just Eco, unless you're within the start/end time in which case it does full export.

I'll keep looking/trying but it does seem a bit of a gap that you can't set the inverter to discharge to load, but not charge unless > AC capacity. Local control mode sounds like it should do it but has no effect.

[gcoan]

The way I stop solar charging on my Gen 1 hybrid (with fast swap firmware) is to set the charge rate to zero or to set battery pause mode to PauseCharging.

The battery mode is set to Eco and the discharge rates are still set to maximum so the battery can discharge to support house load if it needs it, but it won't charge from solar when PauseCharging is set.

I have two inverters so set the one that is connected to the East facing array to a 1kW charge rate in the morning so it allows the battery to trickle charge and reduce solar clipping. The inverter with the West facing array I set to PauseCharging to stop it charging completely.

Then in the afternoon I change the second inverter battery mode from PauseCharging to Disabled so it can charge up with the West solar generation.

Unfortunately I don't know of a 'fire and forget' set of settings but PauseCharging does work well for stopping solar charging

[Philhews]

Yea, I can prevent any battery charging, or slow it but I quite liked stopping it except for excess PV.

[BuhJuhWuh]

Set Charge Low Power Mode might be helpful here...

Does this affect both charging from the grid and from solar? Or just from the grid? I ask because on comparing plans for tomorrow (first forecast sunny day for a few days) with it switched on/off I can see no difference whatsoever between them.

(I guess, given what's been discussed above, that it makes a kind of sense that this would only really affect charging from the grid...?)