egguy / addon-grott-beta

An HA addon base on the 2.8 branch + gott HA plugin

7 stars 1 forks source link

How to install addon-grott-beta in ha ? #31

Closed typxxi closed 3 months ago

typxxi commented 3 months ago

I have opened the addon store, then i have clicked on the 3 dot menu where I chose repository and then pasted in https://github.com/egguy/addon-grott-beta

That did not work. What link do I have to paste there ?

thanks

egguy commented 3 months ago

Hi,

This is the URL for the individual add-on. The add-on follows the use of the HA architecture, which is a git repository per add-on, then a central repository.

For the main repository it is: https://github.com/egguy/ha-addons

typxxi commented 3 months ago

Hi Etienne,

thanks a lot. This worked out for me but it is really, really hard to understand what is needed. I have a tiny bit of understanding that you mean that it depends on which OS I am running HA and that the HA OS requires this ha-addons.

But what is really weird from the outside as a newbie:

1 why has this particular addon NO "GROTT" in its link or name

2 if I click on your name and profile there is no growatt beta mentioned and no ha-addons or i must have overlooked it for hours

For any newbie it is so hard to get that one right with at least the documentation I hand found. And I tried to read all of these documents.

Right now it seems to be running, but no sun here and therefore no access to my smallest stick to change the ip inside which the stick should report to. I have also added 7 inverters in the mapping { "serial": "MIC", "serial": "MIC", "serial": "MIN", "serial": "MIN", "serial": "MIN", "serial": "MIN", "serial": "MIN" } but first the MiC is not mentiones which is the entry line and at least works using grott 2.8.3 on a pi 3 with grott integration.

And then, if MIC is OK to be used this way, what if I also wanna add my SPH 4600 ? Should I append that the same way to the current list ?

I will commission the SPH 4600 at the weekend to check it out.

Thanks a lot, I hope it will work out fine tomorrow when I will move the mic 600 to ha-grott addon.

typxxi commented 3 months ago

I have checked the log / protocoll and it seems to have started and also receiving packets since I moved the first inverter from the dedicated grott proxy to grott on ha.

Based on the ip adress those packets are coming from the inverter I have moved. Then I have checked the growatt web site / app and could see that the data must arrive cause the graphs are continuing steadily, not horizontally, but increasing so data from the inverter are still arriving in the growatt cloud.

But when I checked the grott log in the ha addon AGAIN then i found anerror due to the MiC which I had setup in my mapping sequence. Right now it is the only inverter I have moved. I have not changed anything in the grott.ini of the addon.

On the dedicated pi-3 grott installation the mic were working. I know the MIC are not on your list but I was wondering why 2.8.3. on my pi is able to deal with mic 2000 and mic 600 and why the grott ha addon should not be capable.

     - Matched inverter serial to inverter type MIC
Traceback (most recent call last):
  File "/app/grott.py", line 35, in <module>
    proxy.main(conf)
  File "/app/grottproxy.py", line 144, in main
    self.on_recv(conf)
  File "/app/grottproxy.py", line 238, in on_recv
    procdata(conf,data)    
    ^^^^^^^^^^^^^^^^^^^
  File "/app/grottdata.py", line 220, in procdata
    for keyword in  conf.recorddict[layout].keys() :
                    ~~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'T06NNNNXMIC'
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
....

So I went back to growatt and looked if the data are still coming in even though here it says stopped. Yes, they still seem to arrive in the cloud.

Now I have restarted HA Addon Grott and got the same issue

     - Growatt packet received:
          <socket.socket fd=4, family=2, type=1, proto=0, laddr=('172.30.33.6', 48558), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Matched inverter serial to inverter type MIC
Traceback (most recent call last):
  File "/app/grott.py", line 35, in <module>
    proxy.main(conf)
  File "/app/grottproxy.py", line 144, in main
    self.on_recv(conf)
  File "/app/grottproxy.py", line 238, in on_recv
    procdata(conf,data)    
    ^^^^^^^^^^^^^^^^^^^
  File "/app/grottdata.py", line 220, in procdata
    for keyword in  conf.recorddict[layout].keys() :
                    ~~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'T06NNNNXMIC'
s6-rc: info: service legacy-services: stopping
...

Finally I went back into the configuration and changed the MIC to MIN in the mapping field. And then this happened instead of the error before

     - Growatt packet received:
          <socket.socket fd=4, family=2, type=1, proto=0, laddr=('172.30.33.6', 34852), raddr=('47.91.67.66', 5279)>
     - Growatt data decrypted V2
     - Matched inverter serial to inverter type MIN
     - Grott MQTT topic used : energy/growatt

and grott ha addon keeps running. So far so good, except the sensors created by the grott HA integration by muppet3000 is no longer showing new values, I mean updated values. All data are now about 30 minutes old, while all other inverter have a time stamp from 3 seconds to 1 minute ago.

Here is my addon setup (beyond the mapping fieled with all serials and type of inverters).

For now I will stop the move and return the inverter back to the dedicated grott proxy server on my Pi-3 cause I want the data in HA .

What do I have to change to get the MiC running and get the sensors of this inverter reconnected ? thanks

UPDATE: I have changed the ip adress in the wifi stick back to the pi-3 grott and now the sensors in ha are getting updated every minute again. So it is easy to switch between - unless you have become so "grotty" that you try to use "grott20240315" as the wifi stick password cause that will reply unsuccessfull which took me a bit to figure out why, but grott and growatt are 2 different words even though in the same growatt worlds.

egguy commented 3 months ago

Hi, I just took a look in grott source code (https://github.com/johanmeijer/grott/blob/Beta-(2.8.x)/grottconf.py#L946), there is no mention of the MIC type inverter. Did you add this layout yourself? Or is it a special configuration?

The add-on configuration follows the official configuration, inverter map is invtypemap in the grott.ini, what value did you put in your rpi3 configuration?

If there is missing configuration related to the 2.8 branch of grott, please tell me so I could add it to my addon in case it is missing.

egguy commented 3 months ago

1 why has this particular addon NO "GROTT" in its link or name

The link I gave you is a repository of addons, it is not tied specially to a particular add-on but can contain multiple different add-on (e.g like https://github.com/hassio-addons). It is similar to an APT repository.

This repo follows the official architecture of the HA addons :

A central repo containing all the add-on published (https://github.com/hassio-addons contain a lot) with beta and edge variants if you want to test different versions
1 git repo per add-on. (e.g. this repo: https://github.com/egguy/addon-grott-beta for the beta 2.8 and https://github.com/egguy/addon-grott for the 2.7 , or for HA community addon: adguard: https://github.com/hassio-addons/addon-adguard-home).

As a repository can contain many add-ons, it is not feasible to manage everything at the same place. The separation between add-on repo and the addon allows isolation, bug triage and updates without the creating problems for the unrelated add-ons.

There is a CI/CD pipline ensuring the addon buillds, pass the quality test, and once it is done, it is published to the central repo.

This is why this repo has a mention of grott because it is a repo for HA addons (I'm building other add-ons which will be published in this repo)

2 if I click on your name and profile there is no growatt beta mentioned and no ha-addons or i must have overlooked it for hours

Yeah, sorry, I'm bad at self-promotion :(. As this add-on was mainly related to packaging grott for a friend at the start, I didn't feel the need to mention growatt. I should probably think about this for the future.

Thx for the feedback

typxxi commented 3 months ago

Sorry, I have not that much clue about HA and addon, repositories and so on.

I simply arrived 1 or 2 years ago at grott as the solution for growatt proxy but back then HA had integrated growatt into their core which went well till the api bans kicked in and made it unreliable. At that time we had bought already a lot growatt inverters but had to build the array on our own and rebuild the roof cause during building the solar array I had recogniced water damages on the roof I had to fix first which meant to search, find, buy and pick up old concrete roof tiles of the same era and compliant to the existing one. Therefore a huge delay and not much time to get further into HA.

But right now with more time I digged into that, went the easy way with setting up a dedicated grott server and then mqtt and finally the integration. But I had starred your solition for more than a year as it felt the way to go as a HA OS user if I wanted to avoid the risk and challenges if I would install grott via terminal under HA OS. And since then I followed the developement from time to time but did not get any further cause I had not even setup my grott machine which had happened last week and finally went well.

And then I arrived again in a kind of djungle considering how much is called grott and what it does. here and addon there an integration and then a plugin next to a server , it has become a nightmare from the outside, but my understand is so limited that I can not tell you how to do anything better.

I might write a documentation for those in my situation with running HA OS on a pi whatsoever and the goal to integrate grott on that machine and how to achieve that. Just that path, nothing else cause I bet there are a lot more ways considering that Johann had also added descriptions from users of how to integrate or connect his grott server into HA.

So far I like the grott integration cause at the end it worked out and I bet yours will also work out fine once I got beyond the setup / configuration cause I hope to migrate tomorrow a min inverter that should work as MIN 3000 was part of the grott 2.8.3 list. Maybe MIC might come later.

So right now I am following your "friends" path even though I had thought that the majority of growatt ha owners would end up in that way running HA OS on a powerfull Pi 4 or 5 but times are changing, smaller used NUCs and tiny PC have conquered the previously Pi dominated world. Sounds like your friend and I are on niche way if close to no one had asked for that.

I wanted to send a donation but I could not find a link on your profile so maybe time to add that cause I do not need a middleman to take away his share just for sending you money. And due to the fact that you are in the EU using € paypal would not take its toll or share as for developers in the UK where paypal takes a lot. I wanted to send it as family and friend. Would be nice if you can add that option too cause buy me a beer/coffee is not the way we have used before.

thanks for developing this piece for your friend and now for my father who is using the system and needs reliable solutions that work rock solid cause he is 85 years young and driving an EV for almost 5 years. Thanks !

egguy commented 3 months ago

Regarding the MIC inverter type,

could you share your current grott configuration which works with it?

I would like to understand if I missed a 2.8 update or there is a missing configuration in the add-on.

For the PayPal donations, I need to sort this as I'm not living in Europe anymore and you can't have a PayPal account with a different country. You need to close and reopen it...

typxxi commented 3 months ago

Hi Etienne,

regarding MiC 600 and my current running setup off pure grott on a dedicated Pi3 running pi-os 64 bit latest (dec. 2023)

I have downloaded the github files of 2.8.3 (which is a beta if I remember right but it was recommanded by Johann), moved that file into a grott directory under my desktop, unpacked that.

I start grott from the terminal with this command line

sudo python3 grott.py -v

On that Pi I use the grott.ini which I had copied from the /examples directory into my /grott and then edited just these mqqt lines by removing the #

[MQTT]
# Mqtt parameters definitions
# Be aware nomqtt = True means no MQTT processing will be done!!!!!!

nomqtt = False
#ip = localhost
ip = 192.168.178.2
# ip = core-mosquitto  
port = 1883
topic= energy/growatt
auth = True
user = mymqtt
password = mymqtt
#multi inverter Umgebung erfordert dies
mqttinverterintopic = True

Sorry, but I do not know how to copy whole grott.ini via VNC terminal from my Pi therefore just these lines. All other lines are still green and start with a #

From my notes: I must have

1 activated the relevant mqtt lines

2 changed my ip and password

3 changed auth = False to auth = True

4 added the last line for multiinverter mode by paste and copy

BUT I also must admit that during setup I followed Johanns instruction to a certain point and then switched to the HA grott integration provided by muppet3000 And i guess that I came back to johanns description only once regarding the multiinverter support and that was about a yaml file called

template_growatt_eng.yaml

with these lines where a mic 3000 is mentioned by me

# Grott - Home Assistant Growatt sensors 
# 
# This file exposes all sensors from Grott to HA, including dummy sensors for the type of the inverter and the type and serial number of the datalogger (Be aware, the dummy
# sensors have to be set manually) 

- name: Growatt - Type
  unique_id: growatt_invertertype
  # Please set the type of your inverter
  # state: "SPH6000"
  state: "MIC3000"
  icon: mdi:select-inverse

- name: Growatt - Datalogger type
  unique_id: growatt_datloggertype
  # Please set the type of your datalogger
  state: "ShineWiFi-X"
  icon: mdi:select-inverse

- name: Growatt - Datalogger serial
  unique_id: growatt_datlogger_serial
  # Please set the serial number of your datalogger
  state: "123456678"
  icon: mdi:select-inverse

BUT AGAIN: this has been edited only once to setup the first inverter I migrated and accidantly not been touched again. I simply had forgotten about that and migrated my other 7 inverters without adding all these models like mic 2000 or sph 4600 or min 4600 later.

The grott ha integration simply recogniced those mqtt messages the right way.

And here a screenshot of my terminal with all the files but I bet have not changed any other of these files

I hope it helps cause I did nothing else to provide a list of all my inverters, serials, types or what not. Just a plain grott installation , then mqtt setup and grott ha integration installation.

It simply worked at least for me and I can remember - cause I had struggled a lot in the beginning with grott and the auth = false mess - that grott recognised these inverters too and used whatever kind of layout cause I back then had studied all the grott messages in the terminal which I could not when I had used your implementation.

Back then I copied each time a inverter was recogniced by grott for the very first time the lines in the terminal

And therefore I still have the way I did the migration and my very first inverter was a mic 2000

     - Possible Inverter serial abcdefgh
     - Inverter serial not recognised - using inverter type default
     - Growatt new layout processing
         - decrypt       :  True
         - offset        :  6
         - record layout :  T06NNNNX

     - Grott data record date/time processing started
     - date-time:  2024-03-09T17:55:22
     - Grott values retrieved:
         -  datalogserial        :  12345678
         -  pvserial             :  abcdefgh

I hope this will help even though I have not really done anything regarding a kind of "list of inverters and serials and types". It just worked - somehow. My struggles back then were huge , but those had other roots and causes with the multiarchitecture and docker setup that did not work out, then the auth = False default setup which is wrong even though a user and password were the next lines and I couuld not believe that I would have to change that in True.

thanks for digging deeper into that topic.

egguy commented 3 months ago

OK my guess is the MIN inverter type is treated as a generic inverter in your raspi configuration (- Inverter serial not recognised - using inverter type default). The grott error comes from mapping, as there is not existing mapping for MIC.

If you remove every reference to MIC in { "serial": "MIC", "serial": "MIC", "serial": "MIN", "serial": "MIN", "serial": "MIN", "serial": "MIN", "serial": "MIN" } like { "serial": "MIN", "serial": "MIN", "serial": "MIN", "serial": "MIN", "serial": "MIN" } the error should disappear.

If there is not mapping for a serial, grott use the default detection. The inverter mapping allows decoding extra information like the batteries on SPH/SPF model (they need a mapping, without, they appear but have a limited number of information).

Could you try this change?

PS: To see the messages like your deployment of grott, activate the verbose mode

typxxi commented 3 months ago

Hi Etienne, thanks , but it is too late for today cause sun is going down an then the sticks art shutting down.

Oh, no wait a minute. Since yesterday I have commissioned a sph 4600 , which is a growatt hybrid inverter. I am running this right now without a battery cause I am building a 15 kWh LFP one but it is not ready yet.

THerefore I finally succeeded in setting it up running in a usual grid tied inverter mode cause of the box this is not going to happen and the manual is not touching the topic for an average user and reader in any way and I could not get help from growatt (ask the seller) nor the web forums till I found it out yesterday after ... just 21 months and I have already shipped 1 device back to the seller as faulty cause he could not explain it and I told him it is not working.

How complicated that must be to setup a growatt hybrid inverter for use without battery ?

Well, take a piece of even small copper wire and simply connect the 2 bat terminals on the inverter with each others. Suddenly the energy production will begin. Nothing more, just that.

Anyhow: the hybrid inverters are constantly working or at least in standby after sunset and I recogniced tonight mqtt messages arriving which means that this inverter is active even during night.

But this inverter is a complete different beast and already causing issues on the grott integration side cause some values appeared on wrong sensors like a wrong matching table cause I could find the values from the growatt side and then check my hybrid sensors which one was reporting 479 the last time. And then I had the right watt sensor. And I bet that this caused due to the missing mapping on the grott server or the yaml integration cause I had not paid much attention back then when all the mic and min inverters were recogniced by grott and grott ha integration quite right without any issues until I commissioned my sph 4600 yesterday.

Therefore I will try that later tonight or tomorrow during daylight just to check the mic 600 to mic 3000 issue first and not mess things up cause I also have to start the investigation about the SPH 4600 in the current pure grott setup.

Have a nice evening.

egguy commented 3 months ago

Hi,

Both my plugin and muppet integration only work on what has been decoded by grott, if it is incorrect in grott, it will be incorrect in both. By default, mapping the batteries and some sensors are not correctly reported by grott. You need a special translation table for the SPH/SPH.

Here is the way to get the extra sensors:

If you have only one inverter sending data to grott, edit the invtype = default setting and put sph.

In case, you have multiple inverters connected to the same grott. You must keep the invtype to default, but you must setup the invtypemap setting like you have done. Here is the list of supported special mapping: https://github.com/egguy/addon-grott-beta/blob/main/grott-beta/DOCS.md#configuration .

If it is not in the list, making a SERIAL: mapping is going to cause a problem (e.g. MIC)

typxxi commented 3 months ago

Hi Etienne,

I had first to solve some issues I have had with my productive grott, which I use as a sandbox to gain knowledge.

I had gotten - out of nowhere - an error inside grott like mqqt refused data. I had changed seen that before, had then edited grott.ini and for some reason the error had disappeared. Days later the error reappeared, did not go away and caused trouble. I finally changed the mqtt password and - it did not work out. At the end it took me 1 day to get behind. Living in a sandbox I did not start grott via a service, just manually. Life in a sandbox and testing environment. Worked , but at the end it had caused the trouble even though I had started grott accordingly like python /home/pi/grott/grott.py -v

Well, that is causing the trouble. You have to be inside the grott directory otherwise you will get the error refused.

So, I fixed that, added my SPH 4600 to grott ha integration and since then I have a 247 accessible inverter who is creating messages every minute 247 which meant I can test now during night.

And here I am reporting: it is running without any issue and it was totally easy to install cause I guess I did nothing manually.

I have installed it via hac I guess. Went to addon, read your documentation which helped a lot and finally copied my whole list of inverter serials and models, replace mic by min to avoid the previous error. This means that I hope that I will get tomorrow all the data from your grott add on. At least I hope so, but I guess without mic I do not have to fear issues.

I have activated verbose, I check the logs / protocolls and enabled mqtt. I also checked via mqtt explorer what is coming in and what had changed between the messages. Did work out. And it was quite forward and easy if you follow step by stept.

My next steps will be tomorrow when the sun goes up again to migrate my smalles mic inverter (but declared in grott ha as a MiN inverter to avoid any troubles) and then all the others over the weekend.

So far, so good. And I guess it will continue to work this way which means that I would get rid of the current dedicated pi 3 that is only running grott and the desktop, but I think I will use the pi-3 as a backup on the shelf in case things go wrong cause I also have to migrate HA from its Pi-4 4 GB to a new Pi-5 4 GB, both with a SSD.

In case of updates you will migrate Johanns next version into the addon and release it when ready, right ? Johann had announced a beta 3.0 if I remember right.

Looking back it was far easier than thought and so much easier than the manual installation of grott on a dedicated pi ! You can not compare that experience - but you always have to keep in mind if things are not working well or quite right, who is getting the data from whom and where the root might be in case things do not look really right.

But the package of your grott addon and Chris integration is a big winner. The only missing piece is the step by step documentation for a complete beginner. I hope to find some time in the summer to write down what I have done and which steps make the difference to run growatt inverters from a grott ha addon with a grott integration. And there are the points that make it so complicated to understand from the outside and with not much experience. I always have to think twice if I write about grott ha addon and integration and the underlaying grott server which I can only recognise so far by the protocoll tab of your add on. I guess you have to explain in the very first line that based on HA OS you can only run other apps via the add on store just to make clear that the add on is the same as the dedicated grott pi machine I am using so far, but without a container and manual interaction on the command line (except you wanna dig into /config/grott/ whatsoever file). I struggle a lot to get the picture right cause there is a lot going on between stick - grott ha - HA mqtt - HA integration - HA
till finally the growatt inverter data have arrived inside HA in a sensor.

But it works like a charme and it feels like if you have no clue at all what grott is and operates (I mean Johanns version) then it might be a lot easier to start. Otherwise you tend always to compare grott with grott ha even though it will be pretty much the same result cause I could move to grott HA addon easily and also could go back to the old pi 3 grott machine without causing troubles on the grott HA integration and grott HA sensor side.

great achievement !

typxxi commented 3 months ago

learned a lot and everything - that can be fixed (cause a lot like Mic has to be fixed on the grott side first) seems to be fixed right now !