search

s-allius / tsun-gen3-proxy

This proxy enables a reliable connection between TSUN third generation inverters (eg. TSOL MS600, MS800, MS2000) and an MQTT broker to integrate the inverter into typical home automations.
https://s-allius.github.io/tsun-gen3-proxy/
BSD 3-Clause "New" or "Revised" License
40 stars 5 forks source link

Handle catch-up data #72

Open limes007 opened 1 month ago

limes007 commented 1 month ago

Today my inverter was offline for many hours (9:00-18:00) and I noticed some interesting behavior:

When the connection is re-established, the inverter not only sends its current data every 5 minutes. It has been sending 10 additional packets every 5 minutes for ~1 hour. These additional data packets appear to contain data from the time the inverter was offline.

This 'catch-up' data is then used at talent-monitoring.com to fill the gap in the graph for the time the inverter was offline.

Screenshot of 18:30, where data up to 12:30 was subsequently filled. Screenshot_20240514_183038

Screenshot of 19:50, data was subsequently filled completely. Screenshot_20240514_195032

The proxy doesn't distinguish between normal data packets and these catch-up data packets. So all packages are handled as current data, so the chart in Home Assistant looks like this. image

I don't know if it will be possible to process this 'catch-up' data with the correct time stamp and then fill in the data in the Home Assistant. At least it would be good to ignore this data if it cannot be handled appropriately.

s-allius commented 1 month ago

The GEN3PLUS inverter sends the 'catch-up' data with every connection. It would be very easy to send this data to the MQTT broker with the old timestamp. But Home-Assistant only accept data in realtime. My knowledge is, that there is no solution to add the data to the home-assistant database by MQTT. You need an integration running on Home Assistant to write the data into the database. Maybe that is possible with other MQTT clients. Do you have a GEN3PLUS (e.g. MS2000) inverter or one of the GEN3 (e.g MS600) series?

limes007 commented 1 month ago

I understand, I already assumed, that this will be not easy possible.

I have an older model TSOL-MS600, but with probably exotic firmware RSW_400_V1.00.20

s-allius commented 1 month ago

Can you send me a trace or a dump of one of this additional packets with catchup data? Maybe I can add a filter or a timestamp to the MQTT packets

limes007 commented 1 month ago

Of course. I assume, there are sensible private data inside these traces, which should not published. How can I remove them?

s-allius commented 1 month ago

Pls send a mail: catchup@allius.de

s-allius commented 1 month ago

Thank you for the trace. This behavior only seems to be implemented in the rare version SW_400_V1.00.20. However, the GEN3PLUS inverters also do this. In a later version I will include the time stamps in the MQTT messages and install filters in HA via autoconfiguration. Then the data will not be transferred to HA, but anyone using a different solution will be able to evaluate it. Perhaps there is also a solution for retroactively saving the data in the HA database

limes007 commented 1 month ago

Great, thank you.

limes007 commented 2 weeks ago

@s-allius I've been using version 0.8.0 for a few days now, and at the moment I seem to be having a lot of connection problems, so a lot of data is being sent as a catch-up block. Do I need to do some configuration in HA to enable the mentioned filter for the catch-up data? Currently, the chart in HA looks like this. image

s-allius commented 2 weeks ago

Please try the release candidate v0.8.1-rc.1. There is a hot fix which should fix the connection problems for the GEN3 inverters with version 0.8.0.

The filter for the catch-up data in 0.8.0 is only available for GEN3PLUS inverters. For the GEN3 ones I will bring this with version 0.9

limes007 commented 2 weeks ago

Ok, thank you. I'm on v0.8.1-rc.1 now.