data backlog caused by mqtt or grott or the wifi stick?

[typxxi]

I am quite sure this is not an integration issue, but I discovered it now with 240 integrations playing around.

I have 7 inverters next to each other and the values looked not bad, rather plausible.

Until I went into the details and found out that i might have delays according to this

That second last line is showing the "last connection" and there you find values from seconds to 6 hours.

So I went back to mqtt and started the listening option where I found out that messages were arriving constantly but some with quite old time stamps for exampel at 1 pm a message from 27 minutes ago. So I checked the serial and went to growatt web to open the inverter and check the time of the stick but that time was right.

And then I went to the grott server and that system was also punctual, but where can these delays happen or is there an internal queque inside some sticks that try to send a message over and over again till they get a reply ?

And it also feels like that each inverter inside HA is having a constant backlog and is not keeping up which might mean that the sticks are not sending messages from the past - that have been stored - faster to keep up with the current data.

How can they ever get to real time if the send message only once a minute cause if sunset kicks kicks in the inverter will turn off and then the backlog would be carried over to the next morning I guess. Lot of guess work but maybe someone has experienced this issue before.

Thanks

[typxxi]

UPDATE: it seems to me that the sensors is not really updating in the front ui cause now all inverters are 7 hours back, but if I check a sensor then I can see in the lower left corner of the box that it has been updated just a few seconds agos or max 1 min.

Not sure, what is going on cause at the end it seems to work, but the front end status is not shown properly. I have no clue but i am happy that the data are quite right even though maybe time stamps still might be off.

At the end the sensor is callend last data update and therefore it should show the last update as time stamp or how long ago it had been.

[muppet3000]

Hi @typxxi this could be due to the date information that's being pushed into Grott. Sometimes the inverters run on the wrong timezone and therefore the data gets published correctly but with the wrong timestamp. If you check in the grott logs each time a data packet is received it prints out a datestamp with it as well (example below), check what each of your inverters is providing in the grott logs, it's probably that that's you're problem. I think there's a way in grott of just overriding it with your local timezone (I'd need to look for the config option though).

         - MQTT jsonmsg:
                         {"device": "NWCPA47006", "time": "2024-03-19T06:35:46", "buffered": "no",
                         "values": {"datalogserial": "NAC3915162", "pvserial": "NWCPA47006",
                         "pvstatus": 6, "pvpowerin": 0, "pv1voltage": 3451, "pv1current": 0,
                         "pv1watt": 38, "pv2voltage": 1502, "pv2current": 0, "pv2watt": 15,
                         "pvpowerout": 7925, "pvfrequentie": 5003, "pvgridvoltage": 2440,
                         "pvgridcurrent": 34, "pvgridpower": 7918, "pvgridvoltage2": 0,
                         "pvgridcurrent2": 0, "pvgridpower2": 0, "pvgridvoltage3": 0,
                         "pvgridcurrent3": 0, "pvgridpower3": 0, "totworktime": 177588554,
                         "eactoday": 18, "pvenergytoday": 18, "eactotal": 212384, "epvtotal": 164015,
                         "epv1today": 0, "epv1total": 97945, "epv2today": 0, "epv2total": 51345,
                         "pvtemperature": 303, "pvipmtemperature": 285, "pvboosttemp": 275,
                         "bat_dsp": 530, "eacharge_today": 79, "eacharge_total": 73564,
                         "batterytype": 1, "uwsysworkmode": 6, "systemfaultword0": 0,
                         "systemfaultword1": 0, "systemfaultword2": 0, "systemfaultword3": 0,
                         "systemfaultword4": 32, "systemfaultword5": 0, "systemfaultword6": 0,
                         "systemfaultword7": 0, "pdischarge1": 9300, "p1charge1": 0, "vbat": 528,
                         "SOC": 80, "pactouserr": 0, "pactousertot": 0, "pactogridr": 0,
                         "pactogridtot": 0, "plocaloadr": 9300, "plocaloadtot": 9300, "spdspstatus":
                         6, "spbusvolt": 2961, "etouser_tod": 147, "etouser_tot": 172619,
                         "etogrid_tod": 0, "etogrid_tot": 312684, "edischarge1_tod": 22,
                         "edischarge1_tot": 110124, "eharge1_tod": 72, "eharge1_tot": 99749,
                         "elocalload_tod": 97, "elocalload_tot": 34325}}

[typxxi]

Hi Chris @muppet3000 ,

yes and no - as always. I have observed that and had seen the date and time stamp. And also the flag behind that with buffered which could make a difference too "2024-03-19T06:35:46", "buffered": "no"

But my point is the moment in time when this line was created , I mean was your part of the grott terminal created at that time of 06:35:46 ?

In my case I had gotten and read such line at 11:11:00 or so just 5 hours later and I had been wondering , also how that grott buffered does work. Afaik it buffers the data in the stick till it gets reconnected and can stora up to 1 month of data. But how is it keeping up ?

If it connection breaks at 08:00:00 and then at 12:00:00 the connection comes back, what is grott / stick doing ? Sending the messages from 8:01 onwards to grott and every minute just 1 message or will it end in a traffic jam of 240 messages in a queque (cause my wifi stick reporting intervall is 1 minute and therefore 240 messgasges would be buffered). And then that would mean that the 241st message would be the most recent one but the last in the queque.

What I had recognised in my logs just by looking on the grott terminal that a few inverters were behind, but not all and not all the same amount of time. And next time I checked these were no longer behind but others instead.

Again just an observation and I bet this is the wrong spot due to the architecture cause you grott ha integration is just the last mile inside ha, but the backlog must be caused by the stick(s) or grott itself cause I had seen that on the ha mqtt broker / explorer page first, then on grott and therefore it does not belong here but rather to grott itself. But I need to cross check that again and move the topic per copy and paste to grott if it is not caused by the sticks.

So for here and now I will close the issue and say sorry, I could have recogniced that this is the wrong spot, but I was too much in a hurry and did not forget such point (in my huge list of things going on - since we decided to get a new heatpump to lower oil consumption). Thanks !

[muppet3000]

Hi @typxxi

The 1 minute send interval is for latest data, if there is buffered data on the stick it will try and send it as fast as possible to catch up.

All buffered data is accompanied by the "buffered": "yes" message in the Grott logs. Because home assistant doesn't have support for historic data points anything with buffered: yes is dropped by the integration.

The last data update sensor simply pulls the time value from the MQTT message and then displays it in your locale.