Open CarstenGrohmann opened 1 month ago
@CarstenGrohmann hi! This software only publishes the values read from the inverter. Your inverter returns zeros, and that is what is published.
Let's look at this response frame:
a585001015000929acebf602019f7b2f010c0100000000000001037200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000078ab5c15
It contains this ModbusRTU frame inside:
01037200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000078ab
Even without decoding, it is obvious that it contains zeros only. It's worth noting that the CRC is valid, which indicates, that the communication is working correctly.
EDIT: Two questions:
DEYE_FEATURE_SET_TIME
config variable)My converter has no connection to the internet and DEYE_FEATURE_SET_TIME
is set to true
.
I'll update the firmware to the newest 2.32 releae and check, if this solves this issue.
I'm sorry for what is essentially a +1
, but I'm noticing the same behavior.
At some point during the day day_energy
suddenly becomes 0
and then returns to the previous (correct) measurement during the next poll.
Maybe this can be handled so that it simply sends nothing instead of 0
in cases where 0
doesn't make any sense?
@nifoc: Which inverter and which firmware are you using?
Maybe this can be handled so that it simply sends nothing instead of
0
in cases where0
doesn't make any sense?
The question is how to define "where 0
doesn't make any sense"?
Options:
What do you think? Or maybe you have other proposal?
This feature would have to be put behind a feature flag, and be disabled by default.
I had trouble with the newest firmware and rolled back to MW3_16U_5406_1.63. The firmware 1.63 reports the same zero frames as reported above.
Therefore, I would prefer the first option as it prevents data from being pushed like in the "Logger is online for the first time this day" and "First zero value in the evening" sections.
@nifoc: In such a case, are all your values 0 or only day_energy?
Short update as I tested a small hack to filter 0 frames based on our discussion.
I added this temporary workaround to my docker container:
diff --git a/src/deye_modbus.py b/src/deye_modbus.py
index 1178712..13666d4 100644
--- a/src/deye_modbus.py
+++ b/src/deye_modbus.py
@@ -47,6 +47,9 @@ class DeyeModbus:
modbus_crc.reverse()
modbus_resp_frame = self.connector.send_request(modbus_frame + modbus_crc)
+ if "0" * 80 in modbus_resp_frame.hex():
+ self.__log.debug("Ignore empty frame: %s", modbus_resp_frame.hex())
+ return {}
if modbus_resp_frame is None:
return {}
return self.__parse_modbus_read_holding_registers_response(modbus_resp_frame, first_reg, last_reg)
2024-08-31 04:45:50,980 - DeyeModbus - DEBUG - Ignore empty frame: 0103720000000000001292000000000000000000001292000000000000094200000000000000000000138800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000400000000000000000000012a00000000000000000000000000001517
2024-08-31 04:46:51,252 - DeyeModbus - DEBUG - Ignore empty frame: 0103720000000000001292000000000000000000001292000000000000094200000000000000000000138800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000400000000000000000000012a00000000000000000000000000001517
and in combination with a Utility Meter:
utility_meter:
energy:
name: "Production today (daily reset)"
source: sensor.deye_inverter_mqtt_production_today
cycle: daily
Home Assistent generates diagrams without any outliers in the morning or in the afternoon:
As there is no consensus about the best strategy here, my proposal is to implement them one-by-one, as needed, so the end-user can choose the one that suits him best. Yours will be added first @CarstenGrohmann
Let me note that I am experiencing similar sporadic "spikes" readings from Deye SUN12k SG04LP3 in my fork, where I observe regular +ok=... response with valid CRC. I do not use block reads but single parameter and the reads are not zero but usually close to. I thought that some delay (1 second) between AT+INVDATA=... send and +ok=... recv would solve the problem of getting correct value(s) to the logger. So, there is some bug in the firmware that deyecloud.com is resolving somewhat better that us with AT+ protocol and I am looking for a root cause of such behaviour.
@CarstenGrohmann Hi can you add the HA sensor settings of this sensor?
sensor.deye_inverter_mqtt_production_today
I have trouble with my daily sensor values which my small inverter see https://github.com/kbialek/deye-inverter-mqtt/issues/197
I have no special/manual settings for the sensor.deye_inverter_mqtt_production_today
sensor as I use my plugin Deye MQTT HA Plugin to announce the MQTT values to HA. In addition, I use several UtilityMeter sensor is used to reset most values at midnight.
Hi Carsten
I saw that you use a Utility Meter for the daily sensor.
But i guess that doesn't solve my issue that my inverter sends it's last known value (before it got offline) for day_energy
or your production_today
via mqtt when it goes back online,
He resets to 0 when the time is set, but sadly he first sends sensor data via mqqt before that.
I just installed your plugin, but that wouldn't change the issue as it only changes the mqtt topics and auto creates the mqtt sensors (which i have done manually before). Nevertheless a plugin to automatically create the sensors is a great benefit.
But as i said an additional utility meter wouldn't change a thing if the sensor which is used has wrong values. But I will see this tomorrow when the inverter comes back online. I also have to say that we all might have different logger firmware versions. Sadly mine has auto updated to MW3_SSL_5408_1.0B so i can't use the AT Commands any more and have to use the tcp mode. Which might also cause the issue, but i can't downgrade.
The sensor has wrong values from 0:00 - 7:20 which would be then used for the utility meter as well. (image of my day_energy sensor)
Btw do you plan to enable the active_power_regulation
via mqtt as well?
Btw2 why are you using state_class:total
and not total_increasing
for your total / daily sensors?
@CarstenGrohmann
in this forum post they mentioned exactly your issue from your first post and there they had changed the state_class of the mqqt sensor from total
to total_increasing
as i also asked in my post why you use total.
https://www.photovoltaikforum.com/thread/226131-deye-sun-m80-g4-eu-erfahrungen/?postID=3761075#post3761075
@Bhoft My microinverters behave in the same way. I will think how to solve this in a systemic way.
@kbialek good that i am not the only one 👍 I just checked your code and if you aggregrate over multiple inverters you are already checking if now isn't the last aggregation date. https://github.com/kbialek/deye-inverter-mqtt/blob/8a56a28eee905bff2e3b5e0cf9d54b80db76df88/src/deye_multi_inverter_data_aggregator.py#L43-L46
Maybe something like this would also work for single inverter usage?
But as i said an additional utility meter wouldn't change a thing if the sensor which is used has wrong values.
The utility meter cannot change the values of the origin sensor. The utility meter creates a new sensor with changed values and you can use the new sensor instead of the original sensor.
From my perspective the utility meter is a feasible workaround.
Btw do you plan to enable the
active_power_regulation
via mqtt as well?
Yes, I'll do after I added support for another set of topics.
With 3 Utility meters (battery and production) I have been able to correctly restart the sensors at midnight for the Energy card.
Further, I needed to add several sensor templates to correctly recompute for Sunsynk-Power-Flow-Card
where some missing sensors (e.g. frequency) are taken from another inverter (Solaredge).
Btw, I would love to see R/W registers such as peak shaving (191), limit_control (142), (off) grid state, battery limits, ... to be implemented for automated (timed) control of the inverter such as https://github.com/klatremis/esphome-for-deye is!
@CarstenGrohmann
But as i said an additional utility meter wouldn't change a thing if the sensor which is used has wrong values.
The utility meter cannot change the values of the origin sensor. The utility meter creates a new sensor with changed values and you can use the new sensor instead of the original sensor. From my perspective the utility meter is a feasible workaround.
What I meant was if your original sensor sends wrong values in the morning when the inverter comes online the utility sensor will also use these wrong values. But it`s strange that your chart of your utility sensor doesn't show wrong values. Does your daily sensor have wrong values as posts of kbialek and me are showing?
hi!
I'm currently testing this beta image ghcr.io/kbialek/deye-inverter-mqtt:2024.09.1-beta-2
It contains these PRs
I encourage you to give it try
@CarstenGrohmann You will need to add DEYE_DATA_FILTERS=ignore_zeroed_frames
to the config to activate this functionality
Thank you for providing this feature. I've updated the my docker container and share the results during the next days.
What do you think about adding an initial status message like:
DeyeProcessorFactory - INFO - Feature "ignore_zeroed_frames": enabled
I'm currently testing this beta image
ghcr.io/kbialek/deye-inverter-mqtt:2024.09.1-beta-2
It contains these PRs
I have installed the beta version yesterday and have an error in the log (multiple times) but thats now an issue with the Plugin from Carsten with this beta. As now no sensors information is sent to Homeassistent besides application_status
and logger_status
. So the autocreation of sensors via plugin currently isn't working as the plugin needs some code changes.
@CarstenGrohmann
2024-09-13 05:59:04,549 - DeyeDaemon - ERROR - Unexpected error during runner execution
Traceback (most recent call last):
File "/opt/deye_inverter_mqtt/deye_daemon.py", line 53, in __invoke_action
self.__action()
File "/opt/deye_inverter_mqtt/deye_inverter_state.py", line 60, in read_from_logger
processor.process(events)
File "/opt/deye_inverter_mqtt/plugins/deye_plugin_ha_discovery.py", line 237, in process
self.publish_sensor_information(topic, event.observation)
File "/opt/deye_inverter_mqtt/plugins/deye_plugin_ha_discovery.py", line 167, in publish_sensor_information
"unit_of_measurement": observation.sensor.unit,
AttributeError: 'DailyResetSensor' object has no attribute 'unit'
@kbialek Besides that the day energy value is now 0 when the inverter comes online and the first value send is now correct. 👍
2024-09-13 05:10:04,354 - DeyeMqttClient - DEBUG - Publishing message. topic: 'deye/day_energy', value: '0.0'
Same for the PV1-PVx values
2024-09-13 05:10:04,407 - DeyeMqttClient - DEBUG - Publishing message. topic: 'deye/dc/pv1/day_energy', value: '0.0'
Sadly Home Assistant still have the issue with the statistics it saves an value 5 minutes before the mqqt publish changed the value to 0. Just for information the timestamp in the portainer log doesn't have the correct Timezone of Format so 5:10 is 7:10.
Home assistant really saves periodically the current value in the statistics even if no new value is received via MQTT. So through the whole night when the inverter is offline the value isn't changed but saved in the logs.
I guess with "expire_after": 300,
to the mqtt sensor this would be possible to solve.
Which i now have added to my manually created mqqt sensors. I will write something tomorrow if that helped.
The AttributeError exception is raised from my code. It looks like the unit
attribute is missing from DailyResetSensor
even if the class is derived from Sensor
class.
2024-09-13 11:08:31,622 - DeyeDaemon - ERROR - Unexpected error during runner execution
Traceback (most recent call last):
File "/opt/deye_inverter_mqtt/deye_daemon.py", line 53, in __invoke_action
self.__action()
File "/opt/deye_inverter_mqtt/deye_inverter_state.py", line 60, in read_from_logger
processor.process(events)
File "/opt/deye_inverter_mqtt/plugins/deye_plugin_ha_discovery.py", line 320, in process
self.publish_sensor_information(topic, event.observation)
File "/opt/deye_inverter_mqtt/plugins/deye_plugin_ha_discovery.py", line 242, in publish_sensor_information
"unit_of_measurement": observation.sensor.unit,
AttributeError: 'DailyResetSensor' object has no attribute 'unit'
@kbialek: What do you think about this issue?
A new beta image 2024.09.1-beta-3
is there
DeyeDaemon - INFO - Active data filters: ignore_zeroed_frames
I've updated my docker image and published a development branch with new metric and total_increasing state class for day_energy in https://github.com/CarstenGrohmann/deye-mqtt-ha-plugin/tree/more_inverters.
Today it was raining. I'll monitor all changes during the days next week with more sun.
Thank you for this fast update.
I use your software for a few months and it runs stable.
Unfortunatelly it reports unexpected 0s in the morning and the evening e.g. for day_enery and total_energy.
Both zeros are illogical for the total_energy, as the total_energy is a steadily increasing value that should not be 0 except at the very beginning.
The situation is similar with day_energy. Zero values are only expected in the morning after the time reset. In the evening, after the values have risen steadily throughout the day, a 0 value is illogical and unexpected.
I prefer to send no values rather than a series of zeros.
In addition, the 0 for day_energy in the evening breaks the calculation of my self-used solar energy as provided by Home Assistant.
What do you think about this behaviour?
Expected behavior Continuous line that is interrupted overnight because the inverter and logger are offline overnight. The line does not go down to 0 either in the morning or in the evening.
Screenshots
Hardware (please complete the following information):
Software (please complete the following information):
Logs
Logger is online for the first time this day:
First zero value in the morning:
First non-zero value in the morning:
Last non-zero value in the evening:
First zero value in the evening: