[Feature request] Enable MQTT message retain

Freekers commented 1 year ago

The latest version of Grott (2.7.8) supports MQTT message retain. However, enabling this feature does not cover any extensions, such as grott_ha. In order to have MQTT messages sent by grott_ha retained, I think this needs to be added to the code of grott_ha itself. I prefer retained MQTT messages, so that after a reboot of HASS, the last known status is available immediately. In the current situation, my HASS instance shows unknown or unavailable data after a reboot until the inverter sends another data packet.

egguy commented 1 year ago

Hello, I’ve done the modification, I need to do some validation.

I'm working on an automated way to generate the image, so the edge image can be updated automatically.

Freekers commented 1 year ago

Great, thank you so much! I use your image as my daily driver in my setup :)

egguy commented 1 year ago

Hello, Can you pull the image: egguy/grott:current-0.0.7-rc6 and spin it for a test?

I hope I packaged it well, it's my first image with buildx and multi arch (support for raspi). The image is based on python-slim ;)

The new parameter in extvar is : "ha_mqtt_retain": True or False

Thx

Freekers commented 1 year ago

Hey!

I tried the egguy/grott:current-0.0.7-rc6 image

The good news:

Image is much smaller :)
MQTT retain function works!

The bad news:

Data push from grott_ha to HA stopped working after about one hour with the following error: [HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str' I tried restarting the container: same issue I deleted and recreated the container: still the same issue I switched back to the latest image and everything started working again like it should.... Changed back again to current-0.0.7-rc6 and it would fail immediately with the same exception.

Let me know if you need more details, happy to help!

Thanks for your time :)

egguy commented 1 year ago

Thanks a lot for the test. I will try to investigate.

Do you have the message sent just before the error? Was it after the configuration creation? Does the image crash? or it still processes packets but raise an error each time?

Freekers commented 1 year ago

Do you have the message sent just before the error?

Unfortunately I don't. I'm having trouble enabling verbose logging. It seems like this issue is due to the way Grott is packaged in Docker. I already opened an issue at the grott repo for this: https://github.com/johanmeijer/grott/issues/299

Here's all the logging I have:

     **********************************************************************************
     - Grott - libscrc not installed, no CRC checking only record validation on length!
     **********************************************************************************
Grott Growatt logging monitor : 2.7.8
Grott Command line parameters processed:
    verbose:         True
    config file:     grott.ini
    output file:     <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
    nomqtt:          False
    inverterid:      automatic
    pvoutput:        False
    blockcmd:        False
    noipf:           False
Grott process configuration file
Grott process environmental variables
    Pulled 'verbose=None' from the environment
Grott override settings if set in commandline
Grott process json layout files
Grott proxy mode started
     **********************************************************************************
     - Grott - libscrc not installed, no CRC checking only record validation on length!
     **********************************************************************************
Hostname : bad2fd7c52d5
IP :  172.22.0.24 , port :  5279 
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
    Grott HA 0.0.7-rc6 - creating TEG2BXXXXX config in HA, 32 to push
    Pushing 33 configurations payload to HA
    Configurations pushed
[HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str'
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
    Grott HA 0.0.7-rc6 - creating FNH0AXXXXX config in HA, 32 to push
    Pushing 33 configurations payload to HA
    Configurations pushed
[HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str'
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
[HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str'
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
[HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str'
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
[HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str'
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
[HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str'
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34308), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
[HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str'
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 35874)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34298), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
[HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str'
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 57082)>

Was it after the configuration creation?

After config creation (see above)

Does the image crash? or it still processes packets but raise an error each time?

No, it does not crash. It still processes packets and raises an error each time

I hope it helps.

Thanks!

egguy commented 1 year ago

I have pushed a new image on the same tag, It has the -v flag removed so you can configure the verbose level.

I've also added the print of the traceback in case of problem, to help me debug.

Freekers commented 1 year ago

I pulled the new image (using the same tag) and did some testing. It looks like the image works if you're not using the new "ha_mqtt_retain": True extvar parameter.

However, if you do use the the new "ha_mqtt_retain": True extvar parameter, the data will not be processed by grott and as such not pushed to HA. This issue occurs instantly after running the image, no need to wait for an hour so. Additionally, the error message is now different. I no longer get [HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str' as error, now the error is as follows: Grott - grottproxy - Invalid data record received, processing stopped for this record

Full logfile:

     **********************************************************************************
     - Grott - libscrc not installed, no CRC checking only record validation on length!
     **********************************************************************************
Grott Growatt logging monitor : 2.7.8
Grott process configuration file
Grott process environmental variables
Grott override settings if set in commandline
Grott process json layout files
Grott proxy mode started
     **********************************************************************************
     - Grott - libscrc not installed, no CRC checking only record validation on length!
     **********************************************************************************
Hostname : 72e4f3183387
IP :  172.22.0.24 , port :  5279 
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 12417)>
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 65480)>
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 12417)>
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 65480)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.195', 12417)>
     - Growatt packet received:
          <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 5279), raddr=('192.168.30.47', 65480)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Inverter serial not recognised - using inverter type default
     - Grott - grottproxy forward error :  [Errno 110] Connection timed out
     - Growatt packet received:
          <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34172), raddr=('47.91.67.66', 5279)>
     - Grott - grottproxy - Invalid data record received, processing stopped for this record
     - Growatt packet received:
          <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 34188), raddr=('47.91.67.66', 5279)>
     - Grott - grottproxy - Invalid data record received, processing stopped for this record
     - Growatt packet received:
          <socket.socket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 47976), raddr=('47.91.67.66', 5279)>
     - Grott - grottproxy - Invalid data record received, processing stopped for this record
     - Growatt packet received:
          <socket.socket fd=10, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 39006), raddr=('47.91.67.66', 5279)>
     - Grott - grottproxy - Invalid data record received, processing stopped for this record
     - Growatt packet received:
          <socket.socket fd=14, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 39030), raddr=('47.91.67.66', 5279)>
     - Grott - grottproxy - Invalid data record received, processing stopped for this record
     - Growatt packet received:
          <socket.socket fd=16, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 39042), raddr=('47.91.67.66', 5279)>
     - Grott - grottproxy - Invalid data record received, processing stopped for this record
     - Growatt packet received:
          <socket.socket fd=20, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 39058), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=22, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 39074), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=20, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 39058), raddr=('47.91.67.66', 5279)>
     - Growatt packet received:
          <socket.socket fd=22, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.22.0.24', 39074), raddr=('47.91.67.66', 5279)>

As you can see from my docker-compose below, I enabled verbose logging but I still don't see it in the container's log:

services:
    grott:
        image: egguy/grott:current-0.0.7-rc6
        container_name: grott
        restart: unless-stopped
        ports:
            - "5279:5279"

        environment:
            gverbose: True
            gnomqtt: True
            gextension: True
            gextname: grott_ha
            gextvar: '{"ha_mqtt_host": "172.22.0.252", "ha_mqtt_port": "1883", "ha_mqtt_user": "XXXXX", "ha_mqtt_password": "XXXXXXX", "ha_mqtt_retain": True}'
            TZ: Europe/Amsterdam

Thanks!

egguy commented 1 year ago

So for the Invalid data record received, processing stopped for this record, this is a grott problem, this function is called before the plugin.

I also had a similar output yesterday. It seems growatt server has some problems, and the inverter had problem connecting, you can see it with the line: - Grott - grottproxy forward error : [Errno 110] Connection timed out line

I don't know if it's related but yesterday, I restarted the container multiples times before, this could be a rate-limiting function from growatt. After stopping for 2 minutes, it restarted processing, but it was mostly buffered.

When this happens, the stick often store old value, and replay it until it's processed by the server. In the meantime, you don't see any data in HA because I don't process buffered records (HA doesn't seem to have a function to send old state, but I can be wrong).

And for the verbose mode, I've found the bug, you need to set the env value as:

      verbose: True
      gverbose: True

I have opened an MR on grott to correct this behaviour. I will try to release a needed version with an enhanced debugging function (better traceback to help diagnose problems)

Freekers commented 1 year ago

Thanks for the information regarding the Verbose envars. I can confirm that verbose logging now works by using/setting both the aforementioned envars. I.e. you indeed need both verbose and gverbose for verbose logging to work.

However, I'm not sure if anything else has changed in the image, but I have not receieved the [HA ext] - Exception while publishing - unsupported operand type(s) for |: 'int' and 'str' error anymore. It has been running stable the past 2 days with "ha_mqtt_retain": True.

I suggest to close this issue. Should the error return, I will open a new issue with vebose logging.

Thanks again!

egguy commented 1 year ago

The feature has been implemented in the -current version of the addon. I'm closing it at the moment.

egguy / grott-home-assistant-add-on

[Feature request] Enable MQTT message retain #12