energy dashboard giving wrong results

[drsmarsden]

config as per doc running 2022.7.1

debug log just has initial subscription logging

Stuart

[drsmarsden]

I found .storage/energy which allows me to map entities to the charts. I have added the raw entities to HA

just looking at the first one sensor.solis_inverter_energy_today is showing 12.2kWh the energy dashboard is showing 55.2kWh

I shall check the rest

{ "version": 1, "minor_version": 1, "key": "energy", "data": { "energy_sources": [ { "type": "solar", "stat_energy_from": "sensor.solis_inverter_energy_today", "config_entry_solar_forecast": null }, { "type": "battery", "stat_energy_from": "sensor.solis_inverter_daily_energy_discharged", "stat_energy_to": "sensor.solis_inverter_daily_energy_charged" }, { "type": "grid", "flow_from": [ { "stat_energy_from": "sensor.solis_inverter_daily_grid_energy_used", "stat_cost": null, "entity_energy_from": "sensor.solis_inverter_daily_grid_energy_used", "entity_energy_price": null, "number_energy_price": null } ], "flow_to": [ { "stat_energy_to": "sensor.solis_inverter_daily_on_grid_energy", "stat_compensation": null, "entity_energy_to": "sensor.solis_inverter_daily_on_grid_energy", "entity_energy_price": null, "number_energy_price": null } ],
"cost_adjustment_day": 0.0 } ], "device_consumption": [ { "stat_consumption": "sensor.psl_120414_current_energy" } ] }

[hultenvp]

Hi,

Can you make a screenshot of all Solis sensors you do not trust? If like to check if there are inconsistencies in the history graph. Also, could you let me know if you use m.ginlong.com or soliscloud.com?

I've noticed the energy dashboard does some filtering. It might not let go of false data points that easily once it picks them up. I've never looked in the code what they're actually doing, so might be wrong.

Let's first check if the actual sensor data is correct for the whole day and take it from there

Cheers

[drsmarsden]

Hi

will do

using soliscloud

I think "grid consumption" in the dashboard config should be Solis Inverter Daily Grid Energy Purchased - hard to tell as we are self sufficient at the moment

I am going to look for the dashboard code

[hultenvp]

Still, the daily solar production seems to be incorrect.

Not sure if that's because you receive false data or something going on in the energy dash. In soliscloud_api.py line 234 there is a debug line. If you uncomment it it will drop the raw server response in the debug log.

                    #_LOGGER.debug("%s", payload)

Could you post the response from the log here? I can then check if there's something wrong in the response from soliscloud. Make sure you remove any personal info while pasting the log.

[drsmarsden]

Hi

2022-07-20 12:43:15 DEBUG (MainThread) [custom_components.solis.soliscloud_api] {'success': True, 'code': '0', 'msg': '成功', 'data': {'id': ‘xxx, 'userId': ‘xxx’, 'sn': ‘xxx, 'inverterMeterModel': 5, 'collectorsn': '1902716241', 'collectorId': ‘xxx’, 'state': 1, 'stateExceptionFlag': 0, 'collectorName': '', 'simFlowState': -1, 'fullHour': 2.22, 'fullHourStr': 'h', 'currentState': '0', 'warningInfoData': 0, 'timeZone': 0.0, 'daylight': '0', 'model': 'F5', 'productModel': 'F5', 'nationalStandardstr': 'G59/3', 'inverterTemperature': 42.3, 'inverterTemperatureUnit': '℃', 'temp': 105.0, 'tempName': '逆变器运行内部环温', 'stationName': 'MeadeHouse', 'sno': '12709F', 'stationId': 'xxx’, 'version': '160011', 'acOutputType': 1, 'dcInputtype': 1, 'rs485ComAddr': '101', 'dataTimestamp': '1658316980000', 'timeStr': '2022-07-20 11:36:20', 'reactivePower': 2.861, 'apparentPower': 3.019, 'uInitGnd': 0, 'uInitGndStr': 'V', 'dcBus': 0.0, 'dcBusStr': 'V', 'dcBusHalf': 0.0, 'dcBusHalfStr': 'V', 'power': 3.6, 'powerStr': 'kWp', 'powerPec': '1', 'porwerPercent': 0.274, 'pac': 0.987, 'pacStr': 'kW', 'pacPec': '1', 'oneSelf': 980.0, 'eToday': 8.0, 'eTodayStr': 'kWh', 'eMonth': 416.0, 'eMonthStr': 'kWh', 'eYear': 2.78, 'eYearStr': 'MWh', 'eTotal': 15.076, 'eTotalStr': 'MWh', 'uPv1': 183.6, 'uPv1Str': 'V', 'iPv1': 2.3, 'iPv1Str': 'A', 'uPv2': 282.5, 'uPv2Str': 'V', 'iPv2': 2.0, 'iPv2Str': 'A', 'uPv3': 0, 'uPv3Str': 'V', 'iPv3': 0, 'iPv3Str': 'A', 'uPv4': 0, 'uPv4Str': 'V', 'iPv4': 0, 'iPv4Str': 'A', 'uPv5': 0, 'uPv5Str': 'V', 'iPv5': 0, 'iPv5Str': 'A', 'uPv6': 0, 'uPv6Str': 'V', 'iPv6': 0, 'iPv6Str': 'A', 'uPv7': 0, 'uPv7Str': 'V', 'iPv7': 0, 'iPv7Str': 'A', 'uPv8': 0, 'uPv8Str': 'V', 'iPv8': 0, 'iPv8Str': 'A', 'uPv9': 0, 'uPv9Str': 'V', 'iPv9': 0, 'iPv9Str': 'A', 'uPv10': 0, 'uPv10Str': 'V', 'iPv10': 0, 'iPv10Str': 'A', 'uPv11': 0, 'uPv11Str': 'V', 'iPv11': 0, 'iPv11Str': 'A', 'uPv12': 0, 'uPv12Str': 'V', 'iPv12': 0, 'iPv12Str': 'A', 'uPv13': 0, 'uPv13Str': 'V', 'iPv13': 0, 'iPv13Str': 'A', 'uPv14': 0, 'uPv14Str': 'V', 'iPv14': 0, 'iPv14Str': 'A', 'uPv15': 0, 'uPv15Str': 'V', 'iPv15': 0, 'iPv15Str': 'A', 'uPv16': 0, 'uPv16Str': 'V', 'iPv16': 0, 'iPv16Str': 'A', 'uPv17': 0, 'uPv17Str': 'V', 'iPv17': 0, 'iPv17Str': 'A', 'uPv18': 0, 'uPv18Str': 'V', 'iPv18': 0, 'iPv18Str': 'A', 'uPv19': 0, 'uPv19Str': 'V', 'iPv19': 0, 'iPv19Str': 'A', 'uPv20': 0, 'uPv20Str': 'V', 'iPv20': 0, 'iPv20Str': 'A', 'uPv21': 0, 'uPv21Str': 'V', 'iPv21': 0, 'iPv21Str': 'A', 'uPv22': 0, 'uPv22Str': 'V', 'iPv22': 0, 'iPv22Str': 'A', 'uPv23': 0, 'uPv23Str': 'V', 'iPv23': 0, 'iPv23Str': 'A', 'uPv24': 0, 'uPv24Str': 'V', 'iPv24': 0, 'iPv24Str': 'A', 'uPv25': 0, 'uPv25Str': 'V', 'iPv25': 0, 'iPv25Str': 'A', 'uPv26': 0, 'uPv26Str': 'V', 'iPv26': 0, 'iPv26Str': 'A', 'uPv27': 0, 'uPv27Str': 'V', 'iPv27': 0, 'iPv27Str': 'A', 'uPv28': 0, 'uPv28Str': 'V', 'iPv28': 0, 'iPv28Str': 'A', 'uPv29': 0, 'uPv29Str': 'V', 'iPv29': 0, 'iPv29Str': 'A', 'uPv30': 0, 'uPv30Str': 'V', 'iPv30': 0, 'iPv30Str': 'A', 'uPv31': 0, 'uPv31Str': 'V', 'iPv31': 0, 'iPv31Str': 'A', 'uPv32': 0, 'uPv32Str': 'V', 'iPv32': 0, 'iPv32Str': 'A', 'pow1': 422, 'pow1Str': 'W', 'pow2': 565, 'pow2Str': 'W', 'pow3': 0, 'pow3Str': 'W', 'pow4': 0, 'pow4Str': 'W', 'pow5': 0, 'pow5Str': 'W', 'pow6': 0, 'pow6Str': 'W', 'pow7': 0, 'pow7Str': 'W', 'pow8': 0, 'pow8Str': 'W', 'pow9': 0, 'pow9Str': 'W', 'pow10': 0, 'pow10Str': 'W', 'pow11': 0, 'pow11Str': 'W', 'pow12': 0, 'pow12Str': 'W', 'pow13': 0, 'pow13Str': 'W', 'pow14': 0, 'pow14Str': 'W', 'pow15': 0, 'pow15Str': 'W', 'pow16': 0, 'pow16Str': 'W', 'pow17': 0, 'pow17Str': 'W', 'pow18': 0, 'pow18Str': 'W', 'pow19': 0, 'pow19Str': 'W', 'pow20': 0, 'pow20Str': 'W', 'pow21': 0, 'pow21Str': 'W', 'pow22': 0, 'pow22Str': 'W', 'pow23': 0, 'pow23Str': 'W', 'pow24': 0, 'pow24Str': 'W', 'pow25': 0, 'pow25Str': 'W', 'pow26': 0, 'pow26Str': 'W', 'pow27': 0, 'pow27Str': 'W', 'pow28': 0, 'pow28Str': 'W', 'pow29': 0, 'pow29Str': 'W', 'pow30': 0, 'pow30Str': 'W', 'pow31': 0, 'pow31Str': 'W', 'pow32': 0, 'pow32Str': 'W', 'uAc1': 247.5, 'uAc1Str': 'V', 'iAc1': 6.1, 'iAc1Str': 'A', 'uAc2': 0.0, 'uAc2Str': 'V', 'iAc2': 0.0, 'iAc2Str': 'A', 'uAc3': 0.0, 'uAc3Str': 'V', 'iAc3': 0.0, 'iAc3Str': 'A', 'powerFactor': 1.0, 'batteryDischargeEnergy': 0, 'batteryDischargeEnergyStr': 'kWh', 'batteryChargeEnergy': 0, 'batteryChargeEnergyStr': 'kWh', 'homeLoadEnergy': 0, 'homeLoadEnergyStr': 'kWh', 'gridPurchasedEnergy': 0, 'gridPurchasedEnergyStr': 'kWh', 'gridSellEnergy': 0, 'gridSellEnergyStr': 'kWh', 'fac': 49.93, 'facStr': 'Hz', 'batteryPower': -0.608, 'batteryPowerStr': 'kW', 'batteryPowerPec': '1', 'batteryPowerZheng': 0, 'batteryPowerFu': 608.0, 'storageBatteryVoltage': 49.9, 'storageBatteryVoltageStr': 'V', 'storageBatteryCurrent': -12.2, 'storageBatteryCurrentStr': 'A', 'batteryCapacitySoc': 59.0, 'batteryHealthSoh': 95.0, 'batteryVoltage': 49.5, 'batteryVoltageStr': 'V', 'bstteryCurrent': 11.6, 'bstteryCurrentStr': 'A', 'batteryPowerBms': 0, 'batteryPowerBmsStr': 'kW', 'batteryChargingCurrent': 100.0, 'batteryChargingCurrentStr': 'A', 'batteryDischargeLimiting': 100.0, 'batteryDischargeLimitingStr': 'A', 'batteryFailureInformation01': '0', 'batteryFailureInformation02': '0', 'batteryTotalChargeEnergy': 5.38, 'batteryTotalChargeEnergyStr': 'MWh', 'batteryTodayChargeEnergy': 4.3, 'batteryTodayChargeEnergyStr': 'kWh', 'batteryMonthChargeEnergy': 0, 'batteryMonthChargeEnergyStr': 'kWh', 'batteryYearChargeEnergy': 0, 'batteryYearChargeEnergyStr': 'kWh', 'batteryYesterdayChargeEnergy': 6.4, 'batteryYesterdayChargeEnergyStr': 'kWh', 'batteryTotalDischargeEnergy': 5.584, 'batteryTotalDischargeEnergyStr': 'MWh', 'batteryTodayDischargeEnergy': 0.8, 'batteryTodayDischargeEnergyStr': 'kWh', 'batteryMonthDischargeEnergy': 0, 'batteryMonthDischargeEnergyStr': 'kWh', 'batteryYearDischargeEnergy': 0, 'batteryYearDischargeEnergyStr': 'kWh', 'batteryYesterdayDischargeEnergy': 11.9, 'batteryYesterdayDischargeEnergyStr': 'kWh', 'gridPurchasedTotalEnergy': 23.234, 'gridPurchasedTotalEnergyStr': 'MWh', 'gridPurchasedYearEnergy': 0, 'gridPurchasedYearEnergyStr': 'kWh', 'gridPurchasedMonthEnergy': 0, 'gridPurchasedMonthEnergyStr': 'kWh', 'gridPurchasedTodayEnergy': 3.3, 'gridPurchasedTodayEnergyStr': 'kWh', 'gridPurchasedYesterdayEnergy': 0.0, 'gridPurchasedYesterdayEnergyStr': 'kWh', 'gridSellTotalEnergy': 1.009, 'gridSellTotalEnergyStr': 'MWh', 'gridSellYearEnergy': 0, 'gridSellYearEnergyStr': 'kWh', 'gridSellMonthEnergy': 0, 'gridSellMonthEnergyStr': 'kWh', 'gridSellTodayEnergy': 0.0, 'gridSellTodayEnergyStr': 'kWh', 'gridSellYesterdayEnergy': 0.0, 'gridSellYesterdayEnergyStr': 'kWh', 'homeLoadTotalEnergy': 37.572, 'homeLoadTotalEnergyStr': 'MWh', 'homeLoadTodayEnergy': 8.1, 'homeLoadTodayEnergyStr': 'kWh', 'homeLoadYesterdayEnergy': 31.1, 'homeLoadYesterdayEnergyStr': 'kWh', 'familyLoadPower': 1.448, 'familyLoadPowerStr': 'kW', 'familyLoadPercent': 0, 'bypassLoadPower': 0.0, 'bypassLoadPowerStr': 'kW', 'bypassAcVoltage': 248.0, 'bypassAcVoltageB': 0.0, 'bypassAcVoltageC': 0.0, 'bypassAcCurrent': 0.0, 'bypassAcCurrentB': 0.0, 'bypassAcCurrentC': 0.0, 'batteryType': '0.0', 'socDischargeSet': 0.0, 'socChargingSet': 0.0, 'epmFailSafe': 0.0, 'psumCalPec': '1', 'insulationResistance': 0.0, 'dispersionRate': 0.0, 'sirRealtime': 0, 'iLeakLimt': 0, 'upvTotal': 0, 'upvTotalStr': 'V', 'ipvTotal': 0, 'ipvTotalStr': 'A', 'powTotal': 0, 'powTotalStr': 'W', 'parallelStatus': 0, 'parallelAddr': 0, 'parallelPhase': 0, 'parallelBattery': 0, 'batteryAlarm': '0', 'batteryCDEnableSet': 0, 'batteryCDSet': 0, 'batteryCDISet': 0, 'batteryCMaxiSet': 0, 'batteryDMaxiSet': 0, 'batteryUvpSet': 0, 'batteryFcvSet': 0, 'batteryAcvSet': 0, 'batteryOvpSet': 0, 'batteryLaTemp': 0, 'pepmSet': 0.0, 'pepm': 0.0, 'psum': 0.007, 'psumCal': 0.007, 'familyLoadPowerPec': '1', 'pepmSetStr': 'kW', 'pepmStr': 'kW', 'reactivePowerStr': 'kVar', 'apparentPowerStr': 'kVA', 'psumStr': 'kW', 'psumCalStr': 'kW'}} 2022-07-20 12:43:15 DEBUG (MainThread) [custom_components.solis.service] Scheduling next update in 2 minutes.

Increased our consumption so total imported none zero

Stuart

On 20 Jul 2022, at 10:01, hultenvp @.***> wrote:

Still, the daily solar production seems to be incorrect.

Not sure if that's because you receive false data or something going on in the energy dash. In soliscloud_api.py line 234 there is a debug line. If you uncomment it it will drop the raw server response in the debug log.

                #_LOGGER.debug("%s", payload)

Could you post the response from the log here? I can then check if there's something wrong in the response from soliscloud. Make sure you remove any personal info while pasting the log.

— Reply to this email directly, view it on GitHub https://github.com/hultenvp/solis-sensor/issues/128#issuecomment-1190016837, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV53RLU4K4WOXCH7YD55TTLVU657FANCNFSM5335GY6A. You are receiving this because you authored the thread.

[hultenvp]

I cannot really assess the battery related figures, but the rest of the data at least looks reasonable. What I do notice is that daylight = 0. Could it be that daylight saving time is not factored in, causing the figures of previous day to spill over into the new day? Best way to check that is to look at energy today history graph. It should nicely reset at midnight. I've seen timezone issues both in the server, as well as in the inverter settings itself causing issues in energy dashboard stats. Added screenshot of my setup showing the expected behavior.

[drsmarsden]

Hi

Some progress

The numbers are being reset at midnight , but there is a spike equal to yesterdays production at 1 am

So if you add 14kWh from yesterday, to todays hourly values as shown in HA, you get 38.5 kWh shown in the top circle. However these values are all wrong - we have a max capacity of 5kWh

Today is cloudy - soliscloud app is showing total today of 7.5kWh

It is hard to be sure, I think battery and grid are both measured directly and appear close.

I assume Home is computed from the other values, so if solar is wrong , home will be wrong

I cannot see any setting for DST, but I can currently login into soliscloud.com http://soliscloud.com/ get "query: more than one user” I have raised a ticket with eu-support, but not holding my breath.

I could do something horrible to get rid of the 1am value, but does not fix the other generation data

Going back to my old physics days , power = I V watts, could there be an incorrect calculation or assumption based on voltage?

Stuart

On 22 Jul 2022, at 07:53, hultenvp @.***> wrote:

I cannot really assess the battery related figures, but the rest of the data at least looks reasonable. What I do notice is that daylight = 0. Could it be that daylight saving time is not factored in, causing the figures of previous day to spill over into the new day? Best way to check that is to look at energy today history graph. It should nicely reset at midnight. I've seen timezone issues both in the server, as well as in the inverter settings itself causing issues in energy dashboard stats. Added screenshot of my setup showing the expected behavior. https://user-images.githubusercontent.com/61835400/180380674-33e81022-0e4c-41ee-bd0d-177595108b46.png — Reply to this email directly, view it on GitHub https://github.com/hultenvp/solis-sensor/issues/128#issuecomment-1192245966, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV53RLUJ7WL4JXR4UU5SFLLVVJAORANCNFSM5335GY6A. You are receiving this because you authored the thread.

[drsmarsden]

Hi

'eToday': 7.8, 'eTodayStr': ‘kWh’ is correct

How is hourly generation calculated ? I assume it must be using a delta from eToday?

Stuart

On 22 Jul 2022, at 14:25, Stuart Marsden @.***> wrote:

Hi

Some progress

<Screenshot 2022-07-22 at 14.09.09.png><Screenshot 2022-07-22 at 14.07.27.png>

The numbers are being reset at midnight , but there is a spike equal to yesterdays production at 1 am

So if you add 14kWh from yesterday, to todays hourly values as shown in HA, you get 38.5 kWh shown in the top circle. However these values are all wrong - we have a max capacity of 5kWh

Today is cloudy - soliscloud app is showing total today of 7.5kWh

It is hard to be sure, I think battery and grid are both measured directly and appear close.

I assume Home is computed from the other values, so if solar is wrong , home will be wrong

I cannot see any setting for DST, but I can currently login into soliscloud.com http://soliscloud.com/ get "query: more than one user” I have raised a ticket with eu-support, but not holding my breath.

I could do something horrible to get rid of the 1am value, but does not fix the other generation data

Going back to my old physics days , power = I V watts, could there be an incorrect calculation or assumption based on voltage?

Stuart

On 22 Jul 2022, at 07:53, hultenvp @. @.>> wrote:

I cannot really assess the battery related figures, but the rest of the data at least looks reasonable. What I do notice is that daylight = 0. Could it be that daylight saving time is not factored in, causing the figures of previous day to spill over into the new day? Best way to check that is to look at energy today history graph. It should nicely reset at midnight. I've seen timezone issues both in the server, as well as in the inverter settings itself causing issues in energy dashboard stats. Added screenshot of my setup showing the expected behavior. https://user-images.githubusercontent.com/61835400/180380674-33e81022-0e4c-41ee-bd0d-177595108b46.png — Reply to this email directly, view it on GitHub https://github.com/hultenvp/solis-sensor/issues/128#issuecomment-1192245966, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV53RLUJ7WL4JXR4UU5SFLLVVJAORANCNFSM5335GY6A. You are receiving this because you authored the thread.

[drsmarsden]

if attribute == INVERTER_ENERGY_TODAY:

Energy_today is not reset at midnight, but in the morning at

                # sunrise when the inverter switches back on. This messes up the
                # energy dashboard. Return 0 while inverter is still off.
                is_am = datetime.now().hour < 12
                if getattr(data, INVERTER_STATE) == 2 and is_am:
                    value = 0
                elif getattr(data, INVERTER_STATE) == 1 and is_am:
                    last_updated_state = None
                    try:
                        last_updated_state = \
                            self._subscriptions[serial][INVERTER_STATE].measured
                    except KeyError:
                        pass
                    if last_updated_state is not None:
                        # Hybrid systems do not reset in the morning, but just after midnight.
                        if last_updated_state.hour == 0 and last_updated_state.minute < 15:
                            value = 0
                        # Avoid race conditions when between state change in the morning and
                        # energy today being reset by adding 5 min grace period and skipping update
                        elif last_updated_state + timedelta(minutes=5) > datetime.now():
                            continue

Looks like there is special code for eToday

On 22 Jul 2022, at 14:40, Stuart Marsden @.***> wrote:

Hi

'eToday': 7.8, 'eTodayStr': ‘kWh’ is correct

How is hourly generation calculated ? I assume it must be using a delta from eToday?

Stuart

On 22 Jul 2022, at 14:25, Stuart Marsden @. @.>> wrote:

Hi

Some progress

<Screenshot 2022-07-22 at 14.09.09.png><Screenshot 2022-07-22 at 14.07.27.png>

The numbers are being reset at midnight , but there is a spike equal to yesterdays production at 1 am

So if you add 14kWh from yesterday, to todays hourly values as shown in HA, you get 38.5 kWh shown in the top circle. However these values are all wrong - we have a max capacity of 5kWh

Today is cloudy - soliscloud app is showing total today of 7.5kWh

It is hard to be sure, I think battery and grid are both measured directly and appear close.

I assume Home is computed from the other values, so if solar is wrong , home will be wrong

I cannot see any setting for DST, but I can currently login into soliscloud.com http://soliscloud.com/ get "query: more than one user” I have raised a ticket with eu-support, but not holding my breath.

I could do something horrible to get rid of the 1am value, but does not fix the other generation data

Going back to my old physics days , power = I V watts, could there be an incorrect calculation or assumption based on voltage?

Stuart

On 22 Jul 2022, at 07:53, hultenvp @. @.>> wrote:

I cannot really assess the battery related figures, but the rest of the data at least looks reasonable. What I do notice is that daylight = 0. Could it be that daylight saving time is not factored in, causing the figures of previous day to spill over into the new day? Best way to check that is to look at energy today history graph. It should nicely reset at midnight. I've seen timezone issues both in the server, as well as in the inverter settings itself causing issues in energy dashboard stats. Added screenshot of my setup showing the expected behavior. https://user-images.githubusercontent.com/61835400/180380674-33e81022-0e4c-41ee-bd0d-177595108b46.png — Reply to this email directly, view it on GitHub https://github.com/hultenvp/solis-sensor/issues/128#issuecomment-1192245966, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV53RLUJ7WL4JXR4UU5SFLLVVJAORANCNFSM5335GY6A. You are receiving this because you authored the thread.

[drsmarsden]

My inverter_state

On 22 Jul 2022, at 14:52, Stuart Marsden @.***> wrote:

if attribute == INVERTER_ENERGY_TODAY:

Energy_today is not reset at midnight, but in the morning at

                # sunrise when the inverter switches back on. This messes up the
                # energy dashboard. Return 0 while inverter is still off.
                is_am = datetime.now().hour < 12
                if getattr(data, INVERTER_STATE) == 2 and is_am:
                    value = 0
                elif getattr(data, INVERTER_STATE) == 1 and is_am:
                    last_updated_state = None
                    try:
                        last_updated_state = \
                            self._subscriptions[serial][INVERTER_STATE].measured
                    except KeyError:
                        pass
                    if last_updated_state is not None:
                        # Hybrid systems do not reset in the morning, but just after midnight.
                        if last_updated_state.hour == 0 and last_updated_state.minute < 15:
                            value = 0
                        # Avoid race conditions when between state change in the morning and
                        # energy today being reset by adding 5 min grace period and skipping update
                        elif last_updated_state + timedelta(minutes=5) > datetime.now():
                            continue

Looks like there is special code for eToday

On 22 Jul 2022, at 14:40, Stuart Marsden @. @.>> wrote:

Hi

'eToday': 7.8, 'eTodayStr': ‘kWh’ is correct

How is hourly generation calculated ? I assume it must be using a delta from eToday?

Stuart

On 22 Jul 2022, at 14:25, Stuart Marsden @. @.>> wrote:

Hi

Some progress

<Screenshot 2022-07-22 at 14.09.09.png><Screenshot 2022-07-22 at 14.07.27.png>

The numbers are being reset at midnight , but there is a spike equal to yesterdays production at 1 am

So if you add 14kWh from yesterday, to todays hourly values as shown in HA, you get 38.5 kWh shown in the top circle. However these values are all wrong - we have a max capacity of 5kWh

Today is cloudy - soliscloud app is showing total today of 7.5kWh

It is hard to be sure, I think battery and grid are both measured directly and appear close.

I assume Home is computed from the other values, so if solar is wrong , home will be wrong

I cannot see any setting for DST, but I can currently login into soliscloud.com http://soliscloud.com/ get "query: more than one user” I have raised a ticket with eu-support, but not holding my breath.

I could do something horrible to get rid of the 1am value, but does not fix the other generation data

Going back to my old physics days , power = I V watts, could there be an incorrect calculation or assumption based on voltage?

Stuart

On 22 Jul 2022, at 07:53, hultenvp @. @.>> wrote:

I cannot really assess the battery related figures, but the rest of the data at least looks reasonable. What I do notice is that daylight = 0. Could it be that daylight saving time is not factored in, causing the figures of previous day to spill over into the new day? Best way to check that is to look at energy today history graph. It should nicely reset at midnight. I've seen timezone issues both in the server, as well as in the inverter settings itself causing issues in energy dashboard stats. Added screenshot of my setup showing the expected behavior. https://user-images.githubusercontent.com/61835400/180380674-33e81022-0e4c-41ee-bd0d-177595108b46.png — Reply to this email directly, view it on GitHub https://github.com/hultenvp/solis-sensor/issues/128#issuecomment-1192245966, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV53RLUJ7WL4JXR4UU5SFLLVVJAORANCNFSM5335GY6A. You are receiving this because you authored the thread.

[hultenvp]

Hi Stuart

That code filters out silly reset behaviour on m.ginlong.com, but it is in the generic code section. Soliscloud does not have the same problem and you have a hybrid Inverter, so you pass through this section:

                             # Hybrid systems do not reset in the morning, but just after midnight.
                             if last_updated_state.hour == 0 and last_updated_state.minute < 15:
                                 value = 0

Now, if something would be wrong with timezone and/or daylight saving time then you will see the energy today value return zero for 15 minutes after midnight, and then rebound to the old value until the server thinks it is midnight and really reset to zero. By then you have a spike in your energy today that the dashboard happily accepts and messes up your energy today. Please also check locale settings on your Inverter. Got one user that had to fix it there to get everything working as intended.

On the question of the hourly data: energy dashboard calculates hourly figures from the individual energy today samples using the sample timestamps. You could also use yearly energy as source or even lifelong total - as long as the data is incremental - but it's precision it's less than daily. Otherwise I wouldn't have bothered with the reset at midnight fix.

I propose to tackle the energy today issue first before we dive into remaining issues on other values.

Cheers

[hultenvp]

Also have a look at issues #126 and #118

[drsmarsden]

Hi Thanks Now I know my way around the api code a bit will do some experimenting Stuart

Sent from my iPhone

On 22 Jul 2022, at 15:43, hultenvp @.***> wrote:

 Also have a look at issues #126 and #118

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[drsmarsden]

2022-07-22 19:19:39 INFO (MainThread) [custom_components.solis.service] SMSM hour [19] min [19] energy [10.800000] state [1] 2022-07-22 19:21:39 INFO (MainThread) [custom_components.solis.service] SMSM hour [19] min [21] energy [10.800000] state [1] 2022-07-22 19:23:39 INFO (MainThread) [custom_components.solis.service] SMSM hour [19] min [23] energy [10.800000] state [2] 2022-07-22 19:25:40 INFO (MainThread) [custom_components.solis.service] SMSM hour [19] min [25] energy [10.800000] state [1] 2022-07-22 19:27:40 INFO (MainThread) [custom_components.solis.service] SMSM hour [19] min [27] energy [10.800000] state [1] 2022-07-22 19:29:40 INFO (MainThread) [custom_components.solis.service] SMSM hour [19] min [29] energy [10.800000] state [1] 2022-07-22 19:31:40 INFO (MainThread) [custom_components.solis.service] SMSM hour [19] min [31] energy [10.900000] state [1] 2022-07-22 19:33:41 INFO (MainThread) [custom_components.solis.service] SMSM hour [19] min [33] energy [10.900000] state [1] 2022-07-22 19:35:41 INFO (MainThread) [custom_components.solis.service] SMSM hour [19] min [35] energy [10.900000] state [1]

Added some custom logging “SMSM" is a 40 year old tradition of mine so I can grep my logging !

Even with an ethernet (not wifi) logger looks like it can go offline

Will check in the morning

There are no inverter settings for DST that I can find

Thanks for your help so far

Stuart

On 22 Jul 2022, at 17:49, Stuart Marsden @.***> wrote:

Hi Thanks Now I know my way around the api code a bit will do some experimenting Stuart

Sent from my iPhone

On 22 Jul 2022, at 15:43, hultenvp @.***> wrote:



Also have a look at issues #126 https://github.com/hultenvp/solis-sensor/issues/126 and #118 https://github.com/hultenvp/solis-sensor/issues/118 — Reply to this email directly, view it on GitHub https://github.com/hultenvp/solis-sensor/issues/128#issuecomment-1192646599, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV53RLUKBYFVSAKS4OY2FYDVVKXQJANCNFSM5335GY6A. You are receiving this because you authored the thread.

[hultenvp]

The state is also sensor: solis_state. You can use the graphs as well to compare if you want.

Hybrids apparently do not switch off in the night, at least not as long as there's demand on the battery charge. So I do expect state to remain "1" throughout the night in your case.

For the inverter (and web) timezone settings: depending on what libraries the code uses there can be a difference between selecting UTC and UTC+0. If you select UTC then the time will stick to UTC and not take DST into account. If you select UTC+0 then it will take DST into account. I still didn't get around setting up soliscloud myself, so not sure if this does make sense in the soliscloud context, but it might be worth checking as well.

[drsmarsden]

2022-07-22 23:52:20 INFO (MainThread) [custom_components.solis.service] SMSM hour [23] min [52] energy [11.000000] state [2] 2022-07-22 23:54:21 INFO (MainThread) [custom_components.solis.service] SMSM hour [23] min [54] energy [11.000000] state [2] 2022-07-22 23:56:21 INFO (MainThread) [custom_components.solis.service] SMSM hour [23] min [56] energy [11.000000] state [1] 2022-07-22 23:58:21 INFO (MainThread) [custom_components.solis.service] SMSM hour [23] min [58] energy [11.000000] state [1] 2022-07-23 00:00:21 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [0] energy [11.000000] state [2] 2022-07-23 00:00:21 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:02:22 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [2] energy [11.000000] state [2] 2022-07-23 00:02:22 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:04:22 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [4] energy [11.000000] state [1] 2022-07-23 00:04:22 INFO (MainThread) [custom_components.solis.service] SMSM reset 2 2022-07-23 00:06:22 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [6] energy [11.000000] state [1] 2022-07-23 00:06:22 INFO (MainThread) [custom_components.solis.service] SMSM reset 2 2022-07-23 00:08:23 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [8] energy [11.000000] state [1] 2022-07-23 00:08:23 INFO (MainThread) [custom_components.solis.service] SMSM reset 2 2022-07-23 00:10:23 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [10] energy [11.000000] state [2] 2022-07-23 00:10:23 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:12:23 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [12] energy [11.000000] state [1] 2022-07-23 00:12:23 INFO (MainThread) [custom_components.solis.service] SMSM reset 2 2022-07-23 00:14:23 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [14] energy [11.000000] state [1] 2022-07-23 00:14:23 INFO (MainThread) [custom_components.solis.service] SMSM reset 2 2022-07-23 00:16:24 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [16] energy [11.000000] state [1] 2022-07-23 00:16:24 INFO (MainThread) [custom_components.solis.service] SMSM reset 2 2022-07-23 00:18:24 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [18] energy [11.000000] state [2] 2022-07-23 00:18:24 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:20:24 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [20] energy [11.000000] state [1] 2022-07-23 00:22:24 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [22] energy [11.000000] state [1] 2022-07-23 00:24:25 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [24] energy [11.000000] state [1] 2022-07-23 00:26:25 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [26] energy [11.000000] state [2] 2022-07-23 00:26:25 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:28:25 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [28] energy [11.000000] state [1] 2022-07-23 00:30:26 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [30] energy [11.000000] state [1] 2022-07-23 00:32:26 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [32] energy [11.000000] state [1] 2022-07-23 00:34:26 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [34] energy [11.000000] state [2] 2022-07-23 00:34:26 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:36:26 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [36] energy [11.000000] state [2] 2022-07-23 00:36:26 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:38:27 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [38] energy [11.000000] state [1] 2022-07-23 00:40:27 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [40] energy [11.000000] state [1] 2022-07-23 00:42:27 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [42] energy [11.000000] state [2] 2022-07-23 00:42:27 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:44:27 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [44] energy [11.000000] state [2] 2022-07-23 00:44:27 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:46:27 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [46] energy [11.000000] state [1] 2022-07-23 00:48:28 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [48] energy [11.000000] state [1] 2022-07-23 00:50:28 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [50] energy [11.000000] state [1] 2022-07-23 00:52:28 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [52] energy [11.000000] state [1] 2022-07-23 00:54:29 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [54] energy [11.000000] state [1] 2022-07-23 00:56:29 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [56] energy [11.000000] state [2] 2022-07-23 00:56:29 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 00:58:29 INFO (MainThread) [custom_components.solis.service] SMSM hour [0] min [58] energy [11.000000] state [1] 2022-07-23 01:00:29 INFO (MainThread) [custom_components.solis.service] SMSM hour [1] min [0] energy [11.000000] state [1] 2022-07-23 01:02:30 INFO (MainThread) [custom_components.solis.service] SMSM hour [1] min [2] energy [0.000000] state [2] 2022-07-23 01:02:30 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 01:04:30 INFO (MainThread) [custom_components.solis.service] SMSM hour [1] min [4] energy [0.000000] state [2] 2022-07-23 01:04:30 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 01:06:30 INFO (MainThread) [custom_components.solis.service] SMSM hour [1] min [6] energy [0.000000] state [1] 2022-07-23 01:08:31 INFO (MainThread) [custom_components.solis.service] SMSM hour [1] min [8] energy [0.000000] state [1] 2022-07-23 01:10:31 INFO (MainThread) [custom_components.solis.service] SMSM hour [1] min [10] energy [0.000000] state [1] 2022-07-23 01:12:31 INFO (MainThread) [custom_components.solis.service] SMSM hour [1] min [12] energy [0.000000] state [2] 2022-07-23 01:12:31 INFO (MainThread) [custom_components.solis.service] SMSM reset 1 2022-07-23 01:14:31 INFO (MainThread) [custom_components.solis.service] SMSM hour [1] min [14] energy [0.000000] state [1]

My solis state is 1 90% of the time or 2 pattern is same over 24 hours

Resets energy JUSTAFTER midnight UTC

A frig

    for attribute in data.keys():                                                                     
        if attribute in self._subscriptions[serial]:                                                  
            value = getattr(data, attribute)                                                          
            if attribute == INVERTER_ENERGY_TODAY:                                                    
                _LOGGER.info("SMSM hour [%i] min [%i] energy [%f]", datetime.now().hour, datetime.now().minute, value)
                if datetime.now().hour < 3:                                                                                                                     
                    value = 0                                                                                                                                   
                    _LOGGER.info("SMSM reset")                                                                                                                   
            (self._subscriptions[serial][attribute]).data_updated(value, self.last_updated) 

I think would work for me ( we cannot generate power at midnight)

I am not clear how the message timestamp is used, since it is correctly showing UTC time, so HA not using it or assuming DST

So one fix would be: if date from datetime is the next day cf Time stamp then reset value

Will confirm frig tomorrow

Stuart

On 23 Jul 2022, at 08:12, hultenvp @.***> wrote:

The state is also sensor: solis_state. You can use the graphs as well to compare if you want.

Hybrids apparently do not switch off in the night, at least not as long as there's demand on the battery charge. So I do expect state to remain "1" throughout the night in your case. https://user-images.githubusercontent.com/61835400/180594440-b2afcd3c-8069-42dd-813f-b07e165577fb.png For the inverter (and web) timezone settings: depending on what libraries the code uses there can be a difference between selecting UTC and UTC+0. If you select UTC then the time will stick to UTC and not take DST into account. If you select UTC+0 then it will take DST into account. I still didn't get around setting up soliscloud myself, so not sure if this does make sense in the soliscloud context, but it might be worth checking as well.

— Reply to this email directly, view it on GitHub https://github.com/hultenvp/solis-sensor/issues/128#issuecomment-1193076349, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV53RLSN532PFFAXVOAV2R3VVOLMVANCNFSM5335GY6A. You are receiving this because you authored the thread.

[hultenvp]

Interesting.

Could you perhaps also post screenshots of the history of inverter_state and energy_today? Something similar to my recent screenshot. A 12h period from 10pm till 10am would be great. That would capture the reset pattern and the result off the algorithm nicely.

I'd like to see if I can still come up with a generic approach.

[hultenvp]

By the way. Datetime.now() returns local time. Afaik Ha uses UTC internally to avoid issues in time series when DST changes. It transforms timestamps back to local in the UI.

[drsmarsden]

Appears approx correct now with my frig

You can only compare with soliscloud on the hour.

Energy today now picking up my frig

Spotted yesterday soliscloud app displays UTC times (labelled utc+0.00)

Battery % would good option for the energy panel I think

Happy to test any changes

PS do you know how the clear the energy data completely ?

Stuart

On 24 Jul 2022, at 12:06, hultenvp @.***> wrote:

By the way. Datetime.now() returns local time. Afaik Ha uses UTC internally to avoid issues in time series when DST changes. It transforms timestamps back to local in the UI.

— Reply to this email directly, view it on GitHub https://github.com/hultenvp/solis-sensor/issues/128#issuecomment-1193294844, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV53RLSAUJXMBBFUNFLT7S3VVUPUFANCNFSM5335GY6A. You are receiving this because you authored the thread.

[hultenvp]

Nice to hear your fix seems to work for you.

I'm still thinking if I'd like to take it over as is or try to find another way. Problem I see is that some of the Scandinavian users might be preferring a solution that does not reject any (valid) input before 3AM. But at least it would block out any false data and it's a stateless solution (a must to be robust against HA restarts). I'll think about it during my summer break and get back to you in a few weeks, but I like the simplicity.

On the energy data: I recall some comments in one of the recent release notes (may? june?) where they discussed a new feature to be able to edit historical energy data. EDIT: Found it: https://www.home-assistant.io/blog/2022/04/06/release-20224/#adjusting-long-term-statistics

Oh, and UTC+0 sounds good, assuming you are in UTC+0 Timezone.

[hultenvp]

Sanity check. What does the DST setting in basic settings say in your config?

[drsmarsden]

Changes reset time between 1am and 2am !

[drsmarsden]

Setting UTC+00:00 which is correct

[hultenvp]

Changes reset time between 1am and 2am !

That's confusing. I'd have expected that with your TZ being GMT+0 that it would change reset time between 23PM local time ( resetting at GMT+0) to 00:00 (GMT+1h DST) local time.

I'm running now with DST on and so not see any odd reset or state behaviour without your frig. But I do not have a hybrid with battery.

[drsmarsden]

Just checked my logs, appears ok now !! I have not changed anything since I did my frig. As you can see from my old log - was not even resetting exactly on hour boundary I non zero sample will screw the results.

hour [0] min [58] energy [11.000000] state [1] hour [1] min [0] energy [11.000000] state [1] hour [1] min [2] energy [0.000000] state [2]

I will see if I can generate an alert if I have to reset to value - and monitor

[hultenvp]

Hi @drsmarsden , are you still interested in this getting fixed or did you move over to a local solution?

[drsmarsden]

Hi

I suggest you close and we can pickup it up again if still an issue when the new api comes available

Thanks

Stuart

On 21 Oct 2022, at 10:08, hultenvp @.***> wrote:

Hi @drsmarsden https://github.com/drsmarsden , are you still interested in this getting fixed or did you move over to a local solution?

— Reply to this email directly, view it on GitHub https://github.com/hultenvp/solis-sensor/issues/128#issuecomment-1286679238, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV53RLRTMFXY32SAQG2CATTWEJMQNANCNFSM5335GY6A. You are receiving this because you were mentioned.

[hultenvp]

Thanks Stuart, closing