Configuration instructions corrections/edits

jkkataja commented 1 year ago

As support for new type inverter has been added (https://github.com/johanmeijer/grott#tl3-x-3-phase-inverter-support) should the instructions also include "tl3" in here https://github.com/muppet3000/homeassistant-grott/blob/main/docs/setup/grott.md ?

Hopefully this helps. Please do tell if you would've preferred this as a pull request or even wait longer before this feedback!

muppet3000 commented 1 year ago

Wow, thanks @jkkataja I wasn't expecting anyone to have even looked at this yet. I'll definitely add that in. I've updated the title of this ticket to be more generic. If you find anything else, feel free to add them here. Does this mean you've actually tried following the instructions or were you just taking a look?

jkkataja commented 1 year ago

Didn’t try them yet, I actually setup Grott to run in a container couple of days ago independently in preparation to be ready when this is ready for testing 😁

muppet3000 commented 1 year ago

That's awesome, I'm going to try and finish the documentation this week. Making this update as-per your suggestion now.

muppet3000 commented 1 year ago

@jkkataja - I think I've finished now. Please go ahead and test the setup guide and give me any feedback you have here.

jkkataja commented 1 year ago

First of all big thanks for your work! Managed to get it up and running and so far looks good. I'll try to include overall ideas / feedback here and you can decide which you want to include and what can be left out (that's my way of saying that a lot of this could be considered as nitpicking also 😅).

It's good that you mentioned about running both integrations simultaneously but would it be a good idea to mention to first disable the old one as opposed to deleting it straight away? That way one can compare and validate the data in a more straightforward way by re-enabling it temporarily (e.g., for me the correct output power sensor was sensor._output_power_2 and it was easier to match from HA directly)
It could be mentioned briefly what the "calculate additional values form raw MQTT values" includes. Also I did enable the checkbox when taking the integration to use, but now when I go to integrations -> define (so I managed to the the picture below 😁) I noticed it shows as unticked, should that be considered as a bug?

Grott setup

I would also mention that if one wants to run the container without managing the configuration file, it can be done using passed environment variables (I'm doing it that way) and mention that those said env variables can be found from the Grott repo (to which you are already linking to). Though, if you consider that you want to to include corresponding env variables along side the configuration file settings I can help with those also.
I would also mention that using authenticated MQTT also works (tested by myself) but might be a good idea to leave the instructions as is. That way those who really want to use authentication probably know what they are doing but they know that they are not wasting time in setting it up.
ip setting -> I would remind that by default docker containers create their own networking so if you have two containers running (Grott and MQTT) they won't be able to communicate if you don't specify the correct IP address (most likely the host machine's address).

Inverter/Data logger configuration

I have ShineWiFi-X dongle and once it is taken in to use, I was not able to log in to directly. What I did to make it use my local Grott instance was:

Login to Growatt website
Under "My Photovoltaic Devices" navigate to "All Device" (do not click inverter settings!)
For me it opens directly the "Data Logger" page, from there select Data Logger settings
Select the "Set Ip" button and enter your local Grott server IP address, the "Please Enter Key To Save" field expects "growatt(currentdate)" example: Today is 2023-05-10 -> value is growatt20230510 and then Save

MQTT

It could be mentioned that for debugging, one can install mosquitto_sub and subscribe to the MQTT server to see if there are any messages being seen. I still have my data logger set to 5 minute interval (might have to look in to how to adjust it to 1 minute from Growatt server route at some point) so I only see a message every 5 minutes. Example command would be mosquitto_sub -h <Grott server IP> -t "energy/growatt" for default values without authentication.

Apart from that, mainly repeating my statement that authenticated version also works and it's good to remember Docker networking stuff.

MQTT Broker

Should the "within a minute" in Summary part be rephrased to something along the lines of "should start appearing as soon as the data logger sends the next update"?

How it works

When one is using dark mode the arrows in the pictures are quite hidden by transparent background

Other

As you were expecting, I can now also see that those fields that were not populated with the old integration are now getting data:

muppet3000 commented 1 year ago

Wow @jkkataja thank you so much for all the feedback. I'm at work for the day now, however I'll go through each of these items tonight and update the relevant parts.

I'm really glad that in the first instance you've got it working, while there are clearly some improvements needed in the documentation, the fact you were able to follow it and get something working is really nice!

Snux commented 1 year ago

Instead of installing grott manually, is there any reason I couldn't use this addon? https://github.com/egguy/grott-home-assistant-add-on

benhumphry commented 1 year ago

@Snux I'm using this without issue

Snux commented 1 year ago

Not directly related, but with all the HA stuff I've been doing I've been mindful of what would happen if for some reason I wasn't around. My wife isn't all technical, so I've always told her that everything HA can be stopped and she can go back to manual for everything just by unplugging the Pi4 which is running it all. Switching to grott kind of breaks that - if we unplug the HA Pi then the shine app will stop working. I might buy a second datalogger and use that for this, then if we needed to go back to standard we could just swap the original datalogger back in.

So I will go ahead and get this new integration running, but wondering if anyone has thoughts on the easiest way to revert it back for a non-tech...

muppet3000 commented 1 year ago

@Snux I'm using this without issue

Hi @benhumphry - Can you confirm for me what you mean here? Have you managed to use my integration to display the data but use the egguy add-on to run Grott itself? If that's what you've done, that's really cool and I'll update the documentation to say that's a supported method.

I don't have the "Supervised" version of Home Assistant so I'm not able to use 'Add-ons' hence why I wrote the setup guide the way that I did, I'll happily update it though if it's a supported/working method - certainly easier for people that have the version of Home Assistant that supports add-ons!

muppet3000 commented 1 year ago

Instead of installing grott manually, is there any reason I couldn't use this addon? https://github.com/egguy/grott-home-assistant-add-on

Waiting for @benhumphry to confirm (see above) but looks like it's possible.

muppet3000 commented 1 year ago

Not directly related, but with all the HA stuff I've been doing I've been mindful of what would happen if for some reason I wasn't around. My wife isn't all technical, so I've always told her that everything HA can be stopped and she can go back to manual for everything just by unplugging the Pi4 which is running it all. Switching to grott kind of breaks that - if we unplug the HA Pi then the shine app will stop working. I might buy a second datalogger and use that for this, then if we needed to go back to standard we could just swap the original datalogger back in.

So I will go ahead and get this new integration running, but wondering if anyone has thoughts on the easiest way to revert it back for a non-tech...

Really good question, in the interests of keeping this thread on-topic, could you raise a new issue for this? I'd love to have a discussion about it.

benhumphry commented 1 year ago

@Snux I'm using this without issue

Hi @benhumphry - Can you confirm for me what you mean here? Have you managed to use my integration to display the data but use the egguy add-on to run Grott itself? If that's what you've done, that's really cool and I'll update the documentation to say that's a supported method.

I don't have the "Supervised" version of Home Assistant so I'm not able to use 'Add-ons' hence why I wrote the setup guide the way that I did, I'll happily update it though if it's a supported/working method - certainly easier for people that have the version of Home Assistant that supports add-ons!

Correct - I'm using egguy's addon and it's working really well with your integration (other than the minor error I reported around discharging unit set to kWh instead of W... ).

Edit: I'm running the 'edge' branch of egguy's addon, because it supports SPH.

muppet3000 commented 1 year ago

Just leaving notes here for myself:

Need to address the feedback from @jkkataja still (these ones)
Need to confirm that SoC is calculated correctly on my own system as it looks like I may have hacked it in some way
Add documentation steps for using the egguy add-on

brhahlen commented 1 year ago

Thanks for this integration. Been setting it up by myself.

Docs look good. One thing that took me way too long is that in the grott.ini, you need to specifically uncomment nomqtt = False, because otherwise grott will not connect to MQTT, for some reason.
What about on running this integration and the offcial (https://www.home-assistant.io/integrations/growatt_server) integration together?
I get two entries for Grott when adding the integration, for some reason. Is that expected?
This is probably more Grott related than on the integration: during setup, my internet connection broke (yay) and that also caused Grott to not send any updates to MQTT anymore.

Everything is set up now, just need to wait for internet to come back to actually see data...

mjooris commented 1 year ago

Thanks for this integration. Been setting it up by myself.

Docs look good. One thing that took me way too long is that in the grott.ini, you need to specifically uncomment nomqtt = False, because otherwise grott will not connect to MQTT, for some reason.

What about on running this integration and the offcial (https://www.home-assistant.io/integrations/growatt_server) integration together?

I get two entries for Grott when adding the integration, for some reason. Is that expected?

This is probably more Grott related than on the integration: during setup, my internet connection broke (yay) and that also caused Grott to not send any updates to MQTT anymore.

Everything is set up now, just need to wait for internet to come back to actually see data...

I had the old (official) and the grot/mqtt integration running together while i was figuring out the new sensor names etc. i had no problems with it. the few values that were exactly the same. it automatically placed a 2 behind it. ex: inverterxxxxx_pv1_power inverterxxxxx_pv1_power2

so I used the ones with 2. when i was done, disabled the old integration and everything works like a charm :D

Snux commented 1 year ago

So I will set this up in a couple of weeks when I get back from holiday because for the first time ever my account got locked. I have a question (actually it's grott specific but is relevant here).

With grott running and sending data to growatt as well as mqtt, can I still set the inverter settings via the phone app or growatt Web site?

I'm going to switch to the new Octopus Flux tariff for the cheap overnight electricity to charge the battery etc, so need to be able to adjust the settings accordingly. Can I still do that?

muppet3000 commented 1 year ago

Hi All, I haven't forgotten about this thread, I've just had a busy week, I'll get around to the documentation updates either tomorrow or next week.

Really glad to hear things are working for people and happy to hear more feedback and act on it where I can.

muppet3000 commented 1 year ago

So I will set this up in a couple of weeks when I get back from holiday because for the first time ever my account got locked. I have a question (actually it's grott specific but is relevant here).

With grott running and sending data to growatt as well as mqtt, can I still set the inverter settings via the phone app or growatt Web site?

I'm going to switch to the new Octopus Flux tariff for the cheap overnight electricity to charge the battery etc, so need to be able to adjust the settings accordingly. Can I still do that?

I still have my system updating using my solar predictor (this one) which uses the Growatt API still as I haven't got around to looking at how I can do it through Grott yet.

With the configuration that I've described in the setup steps the Inverter is still able to receive configuration settings via the Growatt website/app, there is a way to disable that, but I haven't looked into it.

Please reach out if you have any issues/questions.

Snux commented 1 year ago

Apologies for the radio silence :)

This is up and running fine, I just need to spend some time tomorrow switching over the sensor names in some of my dashboards but for the moment I have this and the old integration running together and they seem to live happily. I love all the new sensors too (temp, frequency etc). Nice work.

I bashed my head against a wall for a little while until I realised that I'd installed MQTT but not configured for some reason completely unknown. Duh.

One thing you might want to put in the docs (it really belongs in the grott docs though I suppose) is to maybe configure HA with a fixed IP address. If on next reboot of my router I end up picking up a new IP address then my datalogger won't be sending to the right place as I'm just using DHCP to get an address.

muppet3000 commented 1 year ago

Sorry for the delayed feedback for everything in this chain people - I've been on holiday and had lots of family commitments. I'm now back and actively working on this again.

Thanks @Snux for your feedback and kind words. I'll work out how to update the documentation. With regards to HA having a static IP, I assume you're using the egguy add-on to run Grott in that case? Or do you just have a big shared VM with everything on it. I'm just keen to ensure people need to know that it's Grott, not HA that needs to have the static IP.

Either way, glad to hear things are working for people, I'll update further over the next week or so when I make more progress on bugs.

pmp02 commented 1 year ago

Hi All!

I've been using the old integration (growatt_server) so far and decided to switch to the new Grott integration. I've setup (hopefully) everything except routing the ShineWiFi-X traffic to the Grott server and setting it up in HA. I've went on the docker-compose path (easier, less time consuming). I have a HAOS VM and another VM with docker containers for Grott and MQTT (both static IP).

At this stage, does anyone have some tests I cand run to ensure that the connections are working correctly, before I make the switch in datalogger? If I switch the datalogger now and somehow the data doesn't passthrough Grott to Growatt servers, then it will be a lot more difficult to revert back the datalogger settings :). That's the piece I'm most interested in... to make sure that the route Datalogger -> Grott -> Growatt server will work. The other parts (MQTT and HA) can be troubleshooted (hopefully it won't be the case) without too much pressure.

Thanks in advance.

PS: @muppet3000 Nice work with the integration and putting all pieces together in the documentation! 👌

muppet3000 commented 1 year ago

Hi All!

I've been using the old integration (growatt_server) so far and decided to switch to the new Grott integration. I've setup (hopefully) everything except routing the ShineWiFi-X traffic to the Grott server and setting it up in HA. I've went on the docker-compose path (easier, less time consuming). I have a HAOS VM and another VM with docker containers for Grott and MQTT (both static IP).

At this stage, does anyone have some tests I cand run to ensure that the connections are working correctly, before I make the switch in datalogger? If I switch the datalogger now and somehow the data doesn't passthrough Grott to Growatt servers, then it will be a lot more difficult to revert back the datalogger settings :). That's the piece I'm most interested in... to make sure that the route Datalogger -> Grott -> Growatt server will work. The other parts (MQTT and HA) can be troubleshooted (hopefully it won't be the case) without too much pressure.

Thanks in advance.

PS: @muppet3000 Nice work with the integration and putting all pieces together in the documentation! 👌

Thanks for the feedback, there's still some more stuff I need to add but we're getting there!

The only way to check the datalogger -> grott is to just try it out, so long as you've got the IP correct and the port is open on the VM running grott you'll be fine. You can watch the logs of the grott docker container to see when the datalogger connects. Providing you haven't changed any of the default settings in grott around how it connects to the growatt servers you'll be fine on that front too.

What's the concern about reverting the changes to the datalogger? It should be pretty simple to revert. I need to update the docs with this info, however this issue over here: https://github.com/muppet3000/homeassistant-grott/issues/5#issuecomment-1545710398 is really handy because it turns out you can make this change via the web ui for Growatt server, rather than fiddling around with configuration in the local wifi stick.

Let me know how you get on!

Ultimately, if the data gets into grott and then out to growatt server - you're heading in the right direction!

pmp02 commented 1 year ago

Thanks @muppet3000 I have made the step :)... Growatt app seems to be ok, I still receive the data in their servers, DataLogger still shows up in my account... so great on this side. 🥳

Now, on HA side, when I've added the Grott integration, when I've done the setup, I've let the default setting for Device Id (which is +, autodetect). After few minutes, it autodetected 2 entities (Datalog Serial and PV Serial), but don't see any other entities with values read from the inverter. Did I missed something? When I look into Grott logs, I do see the messages passing through and values read (like voltage_l1, voltage_l2, voltage_l3 ... etc.).

How do I get to see these in HA? Shouldn't the Grott integration create automatically the entities for these as well? Did the previous installation affect it (gowatt_server integration I mean, which I have deleted)?

Thanks!

pmp02 commented 1 year ago

I had a look in sensors_mqtt.py and I only get these 2 entities because there are the only ones that match the mqtt values (datalogserial and pvserial).

Maybe I should add that I have a "MAX/MID/MAC" inverter and a ShineWifi-X datalogger. Is the support for this inverter not added yet (I thought I read some weeks back that this is also supported now)?

These are the other values I get from Grott but not reaching HA, if it helps:

        "voltage_l1": 2363,
        "voltage_l2": 2349,
        "voltage_l3": 2368,
        "Current_l1": 106,
        "Current_l2": 69,
        "Current_l3": 121,
        "act_power_l1": -24513,
        "act_power_l2": -15519,
        "act_power_l3": -28821,
        "app_power_l1": 0,
        "app_power_l2": 0,
        "app_power_l3": 0,
        "react_power_l1": -3507,
        "react_power_l2": -2941,
        "react_power_l3": -1520,
        "powerfactor_l1": -971,
        "powerfactor_l2": -958,
        "powerfactor_l3": -998,
        "pos_rev_act_power": -68854,
        "pos_act_power": -68854,
        "rev_act_power": -68854,
        "app_power": 0,
        "react_power": -7969,
        "powerfactor": -979,
        "frequency": 499,
        "L1-2_voltage": 4080,
        "L2-3_voltage": 4085,
        "L3-1_voltage": 4097,
        "pos_act_energy": 10324,
        "rev_act_energy": 32079

muppet3000 commented 1 year ago

Ok, that explains it and that's perfect.

Couple you please help me out and log a new ticket something like "Add support for MAX/MID/MAC systems".

What I need to know is what each of these values actually mean in human terms.

It looks like it's a 3 phase system? So I'm guessing it's something like:

voltage_l1 = Phase 1 - Voltage
voltage_l2 = Phase 2 - Voltage
....

However, I don't know what some of these things mean and what their values are and how much they need to be divided by to be correct. So could you try and help me and give me something like this:

voltage_l1:
- name: Phase 1 - Voltage
  divider: 10 (e.g. 2363 / 10 = 236.2)
  type: voltage

I can guess at some of them but things like powerfactor_l1 I wouldn't want to guess at!

If you could give me a few ideas (just for some key ones) then I could probably throw something together this evening to test.

pmp02 commented 1 year ago

Thought so... :) I'll start new ticket and try to put some descriptions there (try to match against the Growatt app and see if I can get something out of it). Thanks a lot!

muppet3000 commented 1 year ago

No problem, even if you give me just a few that you can guess at, that's a start and I can push something to main for you to test.

Benchamoneh commented 1 year ago

@pmp02 I've got a ShineWifi-S (seems the difference is yours communicates with USB, mine with RS232). I've made the change to my logger several times today however I never see any connections in either my Grott instance or attempting to transit the FW between my logger and Grott server. Is this the setting you changed?

Benchamoneh commented 1 year ago

OK I'm not sure if this is a "feature" of the logging stick, but I moved my Grott VM to the same network as the logger and now I see connections just fine.

It might be worth mentioning that the Grott server seems to need to be local to the logging stick when you update your documentation @muppet3000. I'll do some more testing in the meantime

pmp02 commented 1 year ago

Hi @Benchamoneh , yes, that was the setting. But the format of the key is a bit different (following @jkkataja's explanation above).

https://github.com/muppet3000/homeassistant-grott/issues/1#issuecomment-1541508672

Benchamoneh commented 1 year ago

@pmp02 for me I have to specify the 'growatt' prepend in the mobile app, but in the web interface I can just enter the date and it works. Growatt consistency magic.

I have mine set up and working now and - finally - have data in HA for my SPA3000TL. It took using the beta branch of Grott to get the data correctly but thanks to all involved!

I have noticed a couple of anomolies though so I'll do a quick search and raise a bug if not already present. But I won't derail this issue any more

muppet3000 commented 1 year ago

OK I'm not sure if this is a "feature" of the logging stick, but I moved my Grott VM to the same network as the logger and now I see connections just fine.

It might be worth mentioning that the Grott server seems to need to be local to the logging stick when you update your documentation @muppet3000. I'll do some more testing in the meantime

I'm a little surprised by this given that normally the endpoint is completely outside of the network on the internet i.e. growatt servers. Did you remember to open the access to the port between the two networks?

muppet3000 commented 1 year ago

@pmp02 for me I have to specify the 'growatt' prepend in the mobile app, but in the web interface I can just enter the date and it works. Growatt consistency magic.

I have mine set up and working now and - finally - have data in HA for my SPA3000TL. It took using the beta branch of Grott to get the data correctly but thanks to all involved!

I have noticed a couple of anomolies though so I'll do a quick search and raise a bug if not already present. But I won't derail this issue any more

Glad to hear you got it working, feel free to log another bug if you have any issues.

Benchamoneh commented 1 year ago

OK I'm not sure if this is a "feature" of the logging stick, but I moved my Grott VM to the same network as the logger and now I see connections just fine. It might be worth mentioning that the Grott server seems to need to be local to the logging stick when you update your documentation @muppet3000. I'll do some more testing in the meantime

I'm a little surprised by this given that normally the endpoint is completely outside of the network on the internet i.e. growatt servers. Did you remember to open the access to the port between the two networks?

To come back on this, I suspect it was some random issue with something somewhere as when I moved the container back to its remote VLAN it kept working there too. Whatever it was I can confirm that remote Grott installs are fine

muppet3000 / homeassistant-grott