Cant get ShineWiFi-X to send data to MQTT on home assistant

[lexo12345]

I have Mosquito broker & Grott installed, I have tried setting the IP via wifi hotspot to my home assistant PI's IP address but it kept reverting back to default.

After reading some documentation here: https://github.com/muppet3000/homeassistant-grott/issues/5#issuecomment-1545710398 I noticed that its possible to set via the growatt server. I have set it, but still no data visible on the mosquito log file.

Any idea what I'm doing wrong? Is there a way to check that the datalogger is accepting the IP change after setting via growatt server?

Anything I need to check/any way to be able to test my mosquito MQTT setup?

Any help is much appreciated!

[lexo12345]

Update - managed to get the following in the grott logs, so MQTT must be working. Now to work out why its not displaying as a device/entity

\x6f\x77\x61\x74\x74\x47\x72\x6f\x77\x61\x74\x74\x47\xad\x2b

• Grott automatic protocol detection
• Grott data record length 585
• layout : T060104X
• no matching record layout found, try generic
• Record layout used : T06NNNNX
• Growatt data decrypted V2
• Grott Growatt data decrypted
• Growatt plain data: 030300060241010458474434413530324a4600000000000000000000000000000000000000004 84d47354230363047360000000000000000000000000000000000000000170a03122c15020bb8 0c3400010000000703f8000000000002034000000000000500000000000000000000000000000 000000000000000000000000007138a0965000600000018000000000000000000000000000000 0009650000000000000000000000000000000003f498f2000000f6000311b2000326430000006 8000175d2000000940001b0710000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000f180a00960000000000560000010c00000 000000001670e6300004e20000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000c350cb1000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000001d0006e000f03f70e7003d00000000e000000000000000000000 00000000000df44
• Possible Inverter serial HMG5B060G6
• Inverter serial not recognised - using inverter type default
• Growatt new layout processing
• decrypt : True
• offset : 6
• record layout : T06NNNNX
• Grott data record date/time processing started
• date-time: 2023-10-03T18:44:21
• Grott values retrieved:
• datalogserial : XGD4A502JF
• pvserial : HMG5B060G6
• pvstatus : 1
• pvpowerin : 0.7
• pv1voltage : 101.6
• pv1current : 0.0
• pv1watt : 0.2
• pv2voltage : 83.2
• pv2current : 0.0
• pv2watt : 0.5
• pvpowerout : 0.7
• pvfrequentie : 50.0
• pvgridvoltage : 240.5
• pvgridcurrent : 0.6
• pvgridpower : 2.4
• pvgridvoltage2 : 0.0
• pvgridcurrent2 : 0.0
• pvgridpower2 : 0.0
• pvgridvoltage3 : 0.0
• pvgridcurrent3 : 0.0
• pvgridpower3 : 0.0
• totworktime : 9216.9
• pvenergytoday : 24.6
• pvenergytotal : 20113.8
• epvtotal : 20640.3
• epv1today : 10.4
• epv1total : 9569.8
• epv2today : 14.8
• epv2total : 11070.5
• pvtemperature : 26.8
• pvipmtemperature : 35.9
• MQTT jsonmsg: {"device": "HMG5B060G6", "time": "2023-10-03T18:44:21", "buffered": "no", "values": {"datalogserial": "XGD4A502JF", "pvserial": "HMG5B060G6", "pvstatus": 1, "pvpowerin": 7, "pv1voltage": 1016, "pv1current": 0, "pv1watt": 2, "pv2voltage": 832, "pv2current": 0, "pv2watt": 5, "pvpowerout": 7, "pvfrequentie": 5002, "pvgridvoltage": 2405, "pvgridcurrent": 6, "pvgridpower": 24, "pvgridvoltage2": 0, "pvgridcurrent2": 0, "pvgridpower2": 0, "pvgridvoltage3": 0, "pvgridcurrent3": 0, "pvgridpower3": 0, "totworktime": 66361586, "pvenergytoday": 246, "pvenergytotal": 201138, "epvtotal": 206403, "epv1today": 104, "epv1total": 95698, "epv2today": 148, "epv2total": 110705, "pvtemperature": 268, "pvipmtemperature": 359}}
• Grott MQTT topic used : energy/growatt
• MQTT send failed: [Errno 99] Address not available
• Grott Send data to PVOutput disabled
• Grott Send data to Influx disabled
• Grott extension processing started : grott_ha Grott HA 0.0.7-rc2 - creating HMG5B060G6 config in HA
• Grott extension processing error: ValueError("invalid literal for int() with base 10: ''")
• Traceback (most recent call last): File "/app/grottdata.py", line 677, in procdata ext_result = module.grottext(conf,result_string,jsonmsg) File "/app/grott_ha.py", line 675, in grottext publish_multiple(conf, configs_payloads) File "/app/grott_ha.py", line 589, in publish_multiple conf = process_conf(conf) File "/app/grott_ha.py", line 574, in process_conf port = int(port) ValueError: invalid literal for int() with base 10: ''
• Growatt packet received: <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.30.33.3', 5279), raddr=('192.168.0.102', 6243)>
• Data less then minimum record length, data not processed

[muppet3000]

So how are you running grott? As a standlone process?

You need to remove the 'grott_ha.py' file and completely ignore that.

This integration works by consuming the data from MQTT. Have you installed this integration via HACS into Home Assistant?

Also - Your message above says:

- Grott MQTT topic used : energy/growatt
- MQTT send failed: [Errno 99] Address not available

Which implies that MQTT isn't available for grott to publish to.

[lexo12345]

So how are you running grott? As a standlone process?

You need to remove the 'grott_ha.py' file and completely ignore that.

This integration works by consuming the data from MQTT. Have you installed this integration via HACS into Home Assistant?

Also - Your message above says:

- Grott MQTT topic used : energy/growatt
- MQTT send failed: [Errno 99] Address not available

Which implies that MQTT isn't available for grott to publish to.

I have installed "Grott - HA extension" and I can see what was posted above in the log file. So the data is making its way into the grott HA extension logfile.

Though HACS i have added the repo https://github.com/johanmeijer/grott

However my terminal is telling me that docker & pip3 are not found

[muppet3000]

I'm sorry but in that scenario I don't think there's anything I can do to help. The HA support in the core Grott repo is really poor and the maintainer (johan) didn't add it himself, it's just a PR from someone else. So is grott running as part of Home Assistant? Or have you got it running somewhere else? @egguy - I've just noticed that you made a PR to the core grott repo the other week - does that now add your stand-alone add-on as a core part of grott? Could you help out with the above, I'm really not certain on how it's all supposed to hang together in this scenario.

[lexo12345]

I'm sorry but in that scenario I don't think there's anything I can do to help. The HA support in the core Grott repo is really poor and the maintainer (johan) didn't add it himself, it's just a PR from someone else. So is grott running as part of Home Assistant? Or have you got it running somewhere else? @egguy - I've just noticed that you made a PR to the core grott repo the other week - does that now add your stand-alone add-on as a core part of grott? Could you help out with the above, I'm really not certain on how it's all supposed to hang together in this scenario.

It is installed as per the instructions here: https://github.com/muppet3000/homeassistant-grott

See attached screenshots showing that the data is getting into the grott logs, but not showing as a device/entity.

What am I doing wrong? Ive tried following all the documentation to the best of my ability.

Everything is installed on home assistant on my raspberry pi

[lexo12345]

Problem solved 😃 i am using grott-edge now & have all my data.

Grott HA extension disabled, started grott edge & away we go!

I think I got confused by the different grott services/add ons

[muppet3000]

Yeah that explains it. Glad you got it sorted. I think I need to do a better overview of how to set it up with the @egguy addon and my integration working together. A lot more people are using the version of Home Assistant that supports 'add-ons' than I ever imagined.

[lexo12345]

Yeah that explains it. Glad you got it sorted. I think I need to do a better overview of how to set it up with the @egguy addon and my integration working together. A lot more people are using the version of Home Assistant that supports 'add-ons' than I ever imagined.

Thankyou for your support!

[egguy]

@muppet3000 I'm the maintainer of the extension in grott. The extension was already a part of grott for at least 6+ months, just in a folder called example. I didn't push the change to core until I weeded out the last bug.

I do the packaging of the HA addon to make is easier to set up the extension.

The big difference with my extension is there is no intelligence, it basically translates the raw JSON to the MQTT sensor format for HA.

[martijntonies]

Hi,

Well, I'm quite the newby when it comes to Home Assistant internals. I did have the Growatt integration installed, but as that stopped working, I'm now trying the Grott stuff.

I've installed Grott into Home Assistant, but thats where I got stuck. How do I know it's "running" in any way? I did configure the data logger to point to the HA machine (which is running 'core' or whatever it's called, boots HA automatically, no real access to all files/config etc), but then nothing showed up in the Growatt app/website anymore.

Any points?

[lexo12345]

Hi,

Well, I'm quite the newby when it comes to Home Assistant internals. I did have the Growatt integration installed, but as that stopped working, I'm now trying the Grott stuff.

I've installed Grott into Home Assistant, but thats where I got stuck. How do I know it's "running" in any way? I did configure the data logger to point to the HA machine (which is running 'core' or whatever it's called, boots HA automatically, no real access to all files/config etc), but then nothing showed up in the Growatt app/website anymore.

Any points?

Have you installed grott-edge?

[martijntonies]

Have you installed grott-edge?

No idea what that is ;) I tried following the instructions here > https://github.com/muppet3000/homeassistant-grott/

[lexo12345]

https://github.com/egguy/grott-home-assistant-add-on

Follow these instructions and get the grott edge service running. Worked a charm for me

[martijntonies]

https://github.com/egguy/grott-home-assistant-add-on

Follow these instructions and get the grott edge service running. Worked a charm for me

Hi, thank you for pointing me to that.

When I tried to install the grott add-on from the store, it returned: "The command '/bin/ash -o pipefail -c apk add --update --no-cache python3=3.11.4-r0 && ln -sf python3 /usr/bin/python && python3 -m ensurepip' returned a non-zero code: 1"

Next I tried the edge version, it installed fine. Couldn't get the serial-to-type mapping working with the syntax provider, so instead I entered "sph" in the inverter-type box. Started the add-on, according to the log it's receiving data.

After a few minutes, the HA devices displayed the data as well.

Thanks @lexo12345

Now I have to figure out how to get these back into the Energy dashboard and in my other cards/charts, as all sensors have changed names. Is it possible to create an alias for a sensor?

[muppet3000]

@muppet3000 I'm the maintainer of the extension in grott. The extension was already a part of grott for at least 6+ months, just in a folder called example. I didn't push the change to core until I weeded out the last bug.

I do the packaging of the HA addon to make is easier to set up the extension.

The big difference with my extension is there is no intelligence, it basically translates the raw JSON to the MQTT sensor format for HA.

Ahhh, that makes more sense, so this: https://github.com/egguy/grott-home-assistant-add-on is just a friendly packaged version of this: https://github.com/johanmeijer/grott/tree/master/examples/Home%20Assistent ?

I definitely think we could work together to improve the documentation around this :)

[muppet3000]

https://github.com/egguy/grott-home-assistant-add-on Follow these instructions and get the grott edge service running. Worked a charm for me

Hi, thank you for pointing me to that.

When I tried to install the grott add-on from the store, it returned: "The command '/bin/ash -o pipefail -c apk add --update --no-cache python3=3.11.4-r0 && ln -sf python3 /usr/bin/python && python3 -m ensurepip' returned a non-zero code: 1"

Next I tried the edge version, it installed fine. Couldn't get the serial-to-type mapping working with the syntax provider, so instead I entered "sph" in the inverter-type box. Started the add-on, according to the log it's receiving data.

After a few minutes, the HA devices displayed the data as well.

Thanks @lexo12345

Now I have to figure out how to get these back into the Energy dashboard and in my other cards/charts, as all sensors have changed names. Is it possible to create an alias for a sensor?

That's where my integration, the one that this issue is logged in can help. The edge repo that you're running just pushes data in raw into HA. If you make the following changes when you install it, it'll just push the data to MQTT and then you can use my integration to visualise the data and also create template sensors that re-create your old values (more detail below).

1. Check enable grott MQTT and disable HA integration, then the MQTT devices should be marked as disabled, and grott HACS is going to receive the message.
2. Then install my integration (steps here: https://github.com/muppet3000/homeassistant-grott)
3. All of your entities should then be appearing in the Grott Integration

As for retaining all of your old sensors, I wrote an example/template file here that you can use: https://github.com/muppet3000/homeassistant-grott/blob/main/examples/templates/template_configuration.yaml

[martijntonies]

Hm, first things first.

Not all sensors seem to come through...

The inverter has 2 PV strings, I'm receiving data for both for the actual wattage, but the 'energy today' for string 2 stays at 0.

[muppet3000]

Hm, first things first.

Not all sensors seem to come through...

The inverter has 2 PV strings, I'm receiving data for both for the actual wattage, but the 'energy today' for string 2 stays at 0.

Yesterday you said all the values were coming through. Could you share a screengrab or something? It's not clear what you're referring to.

[martijntonies]

Yesterday you said all the values were coming through. Could you share a screengrab or something? It's not clear what you're referring to.

Yes, I did say that -- what I meant was in the log, all data appeared to be forwarded.

PV1 (string 1) wattage is taken correctly. PV2 (string 2) wattage as well.

In the Energy Dashboard, I had "energy production PV1 today", which seemed to be reset during the night, so that failed. I use "energy product PV1 total" for the stats now, that works.

However, "energy production PV2 today" and "energy production Pv2 total" stays at 0. While the current wattage is correct (different sensor).

The Growatt dashboard on the Growatt website shows the values correctly, but does not list a separate string 1 and string 2 total.

Does this help?

PS: I did not create any templates for this yet nor did I install anything else but Mosquito and the edge-Grott (as far as I remember). Using the sensors as they appear from the MQTT thingy.

[muppet3000]

Yesterday you said all the values were coming through. Could you share a screengrab or something? It's not clear what you're referring to.

Yes, I did say that -- what I meant was in the log, all data appeared to be forwarded.

PV1 (string 1) wattage is taken correctly. PV2 (string 2) wattage as well.

In the Energy Dashboard, I had "energy production PV1 today", which seemed to be reset during the night, so that failed. I use "energy product PV1 total" for the stats now, that works.

However, "energy production PV2 today" and "energy production Pv2 total" stays at 0. While the current wattage is correct (different sensor).

The Growatt dashboard on the Growatt website shows the values correctly, but does not list a separate string 1 and string 2 total.

Does this help?

PS: I did not create any templates for this yet nor did I install anything else but Mosquito and the edge-Grott (as far as I remember). Using the sensors as they appear from the MQTT thingy.

Thanks for the detail, so it looks like one of the sensors isn't correct. Please could you share some of your logs from Grott like I showed above, that way I can look at what Grott thinks the values should be and establish if my integration is misinterpreting them.

[martijntonies]

I enabled debug logging for the MQTT serivce, here's a snippet:

2023-10-07 22:09:08.033 DEBUG (MainThread) [homeassistant.components.mqtt.client] Received message on homeassistant/grott/xxx/state (qos=0): b'{"datalogserial": "xxx", "pvserial": "xxx", "pvstatus": 0, "pvpowerin": 0, "pv1voltage": 564, "pv1current": 0, "pv1watt": 0, "pv2voltage": 561, "pv2current": 0, "pv2watt": 0, "pvpowerout": 0, "pvfrequentie": 0, "pvgridvoltage": 2297, "pvgridcurrent": 0, "pvgridpower": 0, "pvgridvoltage2": 0, "pvgridcurrent2": 0, "pvgridpower2": 0, "pvgridvoltage3": 0, "pvgridcurrent3": 259, "pvgridpower3": 2564620288, "totworktime": 9048, "eactoday": 20365, "pvenergytoday": 20365, "eactotal": 35, "epvtotal": 5898240, "epv1today": 52, "epv1total": 11317, "epv2today": 0, "epv2total": 0, "pvtemperature": 252, "pvipmtemperature": 0, "pvboosttemp": 0, "bat_dsp": 241, "eacharge_today": 0, "eacharge_total": 30, "batterytype": 0, "uwsysworkmode": 0, "systemfaultword0": 0, "systemfaultword1": 0, "systemfaultword2": 0, "systemfaultword3": 0, "systemfaultword4": 0, "systemfaultword5": 0, "systemfaultword6": 0, "systemfaultword7": 0, "pdischarge1": 0, "p1charge1": 0, "vbat": 0, "SOC": 0, "pactouserr": 0, "pactousertot": 0, "pactogridr": 0, "pactogridtot": 0, "plocaloadr": 0, "plocaloadtot": 0, "spdspstatus": 0, "spbusvolt": 0, "etouser_tod": 0, "etouser_tot": 0, "etogrid_tod": 0, "etogrid_tot": 0, "edischarge1_tod": 0, "edischarge1_tot": 0, "eharge1_tod": 0, "eharge1_tot": 0, "elocalload_tod": 0, "elocalload_tot": 0, "grott_last_push": "2023-10-07T20:09:08.024539+00:00"}'

The "incoming payload" has 0 for the value of the PV2 totals... so apparently, the datalogger doesn't send this data like this and the previous integration (the one that pulled the data from the Growatt server) created this data in another manner?

[egguy]

@muppet3000 definitely interested in collaborating together to improve the documentation.

[martijntonies]

@johanmeijer I've looked through the Growatt data in HA and the epv2 data is incorrect. Could this be a record decoding issue?

eactotal: 1 epvtotal: 6553600 // what is this? epv1today: 1 epv1total: 11365 // string 1 total kWh epv2today: 0 epv2total: 0 // string 2, total and today missing

Also, the total energy generated seems to be different from the Growatt Dashboard total

pvenergytoday: 20462 // pvenergy-to-day? Dashboard shows 1975.6 total kWh

Any clues?

[muppet3000]

@johanmeijer I've looked through the Growatt data in HA and the epv2 data is incorrect. Could this be a record decoding issue?

eactotal: 1 epvtotal: 6553600 // what is this? epv1today: 1 epv1total: 11365 // string 1 total kWh epv2today: 0 epv2total: 0 // string 2, total and today missing

Also, the total energy generated seems to be different from the Growatt Dashboard total

pvenergytoday: 20462 // pvenergy-to-day? Dashboard shows 1975.6 total kWh

Any clues?

This isn't the place to log bugs against core grott, you should do that over here: https://github.com/johanmeijer/grott/issues So looking at the logs you posted above this integration is displaying the values that Grott is publishing, but Grott is getting them incorrect?

[johanmeijer]

Please see grott issue #443 (https://github.com/johanmeijer/grott/issues/443).