Limit spurious data spikes on Load and Solar sensors.

[PhilipCraigAU]

G'day Is it possible to set voltage limits on the power sensors. For example if you have 10kW solar inverter it will never exceed say 10,100 watts. Sometimes when I restart my battery I get weird voltage spikes of 100,000 watts + being recorded, which throws out the graphs. Alpha load for house is the same,where the maximum load would never exceed about 11,000 watts in a worse case scenario. Screenshots attached Thanks

[dc42]

I presume you are running the latest version of the ESP software with the CRC issue fixed. I saw a lot of spikes in the data before I fixed that issue. Now I still see occasional spikes in the data. I plan to add some limit checking to the ESP code to ignore readings that are out of range. Alternatively, limit checking could be added to the NodeRed code in the Home Assistant integration.

[PhilipCraigAU]

Thanks for the prompt reply Daniel. The spikes are only occasional, but it only takes one to throw the charts out. I have used the SQLite Web add on and run the following queries to clean up the spikes:-

BOOKMARKS for QUERY

1. delete load > 10,000 DELETE FROM "states" WHERE (attributesid = 9) AND (state LIKE '____%') (ALPHA-LOAD REMOVES POSITIVE VALUES above 10,000)

2. delete load > 100,000 DELETE FROM "states" WHERE (attributes_id = 9) AND (state LIKE '__%') (ALPHA-LOAD REMOVES POSITIVE VALUES above 100,000)

3. delete SOLAR-POWER > 10,000 DELETE FROM "states" WHERE (attributesid = 17) AND (state LIKE '____%') (ALPHA-SOLAR-POWER REMOVES POSITIVE VALUES above 10,000)

I was on V1.25 and have updated to the current V1.26 and re flashed with any updates downloaded in Arduino IDE.

I had a play in the YAML sensor file to try and modify the value template (which gives the correct result in Developer\templates testing area, but it errors in Alpha2MQTT I couldn't work out where in the NodeRed file should I be looking

- device_class: "power"
  state_class: "measurement"
  force_update: "true"
  icon: "mdi:solar-power"
  name: "Alpha Solar Power"
  unique_id: "alpha_solar_power"
  unit_of_measurement: "W"
  state_topic: "HomeAssistant/MQTTSensor/REG_CUSTOM_TOTAL_SOLAR_POWER"
# value_template: "{% if states('sensor.alpha_solar_power') | int <10100 %}{{ value | int ) }}{% else %}{% endif %}}"
  value_template: "{{ value | int }}"  
#  #correct result in Developer\Templates test - but not working here

Greetings from Australia

[jacobcurulli]

Hi @PhilipCraigAU, did you resolve this issue? I'm seeing the same spikes.

[PhilipCraigAU]

@jacobcurulli No it's still unresolved. I'm just running the SQLite queries to clean it up. Note: you have to go into the history to see the clean result. The short term displays the spike annoyingly.

[dc42]

Which values are showing spikes, and for those values what is a reasonable limit above which the reading should be treated as suspicious?

[PhilipCraigAU]

G'day Daniel This relates to Australia if it makes any difference (Voltage is 240v) Ideally these parameters could be set in definitions.h by the user so they could be finessed, or allow for users with higher demands. Otherwise below is probably around the mark.

sensor.alpha_load (House load) would be unlikely to exceed 15,000w
So spikes above 15,000 (or user defined - should be ignored) eg. house base load 500w, aircon flat out 6000w, oven 3500w + 5000w overhead available for additional grid demand.
My grid demand has never exceeded 9.6kWh

sensor.alpha_solar_power (Solar generation) would be unlikely to exceed 15,000w
So spikes above 15,000 (or user defined - should be ignored) eg. the inverters on the Alpha-ESS are typically 5,000w in DC / hybrid mode - Plus any stand alone solar inverters such as mine which are 10,000w. My maximum solar generation is 10,000w as my battery is AC coupled In South Australia households are limited to 5kWh grid export on Single Phase & 15kWh on 3 Phase

Hope this helps. I can send some additional graphs if you require them, and I'm happy to supply further information or be involved in help testing the code. Kind Regards Phil

[dxoverdy]

If you are running the latest version of the code and receiving spikes, then there's something a a little strange going on, because you are getting a large reading AND passing the CRC check.

To get rid of them, I wouldn't go coding it on the chip, instead, amend your Node Red flow to introduce checks with the Switch nodes. Switch nodes are the equivalent of 'IF' statements, allowing you to ask of a payload or other value, "IF X then output via channel 1, IF Y then output via channel 2" In the case of what we want to do here, all we need for battery SOC for example is a single channel output which tests if msg.payload between 0 and 100. As channel 1 is the only output we connect that to the MQTT output. If there is a spurious figure outside 0 and 100 it will just be disregarded. Alternatively you could introduce a channel 2 which sends the message to another MQTT topic where you can log errors.

[PhilipCraigAU]

Thanks Daniel, You gave me enough information to work out what to do, and so far it appears to be working. Thank you so much :)

For those playing at home, here are the screenshots of the modifications to the Node Red flows. Essentially drag the "SWITCH" function between the REG & Send_REG flow, edit with your desired range for maximum house loads and solar inverter production, and rename as desired. Don't forget to deploy.