muppet3000 / homeassistant-grott

Home Assistant Integration for Grott - MQTT
MIT License
71 stars 16 forks source link

Need some help, suddenly sensor unavailable #91

Open thoompje opened 1 month ago

thoompje commented 1 month ago

Hello All,

I am using for a long time now the integration (add-on) but suddenly my sensors getting unavailable. What can cause this issue? First it was just a few times and after a restart of home assistant it was working again, but now I can't get it working again. See screenshots,

image

image

I restarted the inverter also to check if that helped with the issue, but no succes. So hopefully someone can help with the troubleshoot. Many thanks in advance!

muppet3000 commented 1 month ago

There have been a lot of comments on #85 today suggesting that people's wifi sticks got a factory reset overnight. Perhaps the same has happened for you and you need to reconfigure the IP for it to point at Grott?

mjooris commented 1 month ago

mine stopped working to. thx @muppet3000 for the reply. i am going to check this.

thoompje commented 1 month ago

There have been a lot of comments on #85 today suggesting that people's wifi sticks got a factory reset overnight. Perhaps the same has happened for you and you need to reconfigure the IP for it to point at Grott?

Hi, many thanks for the answer en the link to the different threat. It's strange because sometimes it's getting back... But if I configure it now, will it automatically reset again then overnight? Because the inverter is on a difficult place to visit 😅 Or can I configure this on growat.com?

muppet3000 commented 1 month ago

I believe that's what's being discussed over on the other thread. The only way to guarantee that great won't push config to your Wi-Fi stick/dongle is to block incoming traffic from the growatt servers, but then you're unable to push configuration yourself.

A while back I cut out Growatt completely and started using grottserver instead, but I haven't got round to writing a nice python library and HA integration for it yet in order to be able to push config

thoompje commented 1 month ago

I believe that's what's being discussed over on the other thread. The only way to guarantee that great won't push config to your Wi-Fi stick/dongle is to block incoming traffic from the growatt servers, but then you're unable to push configuration yourself.

A while back I cut out Growatt completely and started using grottserver instead, but I haven't got round to writing a nice python library and HA integration for it yet in order to be able to push config

Many thanks again for the answer. If I change the IP on the growat site as before then its all working again... Not happy with them that they change it back to factory. Somebody already contacted the developers or product team?

Screenshot 2024-07-16 at 20 25 22
mjooris commented 1 month ago

Growatt does not want us to do these things, so no use in contacting them.. they'll do anything to keep us from doing this sh*t with me it is working by changing the server.growatt.com back to my grott ip address. but i did notice, when i take the power out of my shine link, the settings go back to growatt.com. thankfully i'm running a pihole dns server in my home so i guess i'll just make a static DNS entry in my dns server that makes server.growatt.com resolve to my internal grott ip address.

87Vortex87 commented 1 month ago

Got the exact same problem today. Really annoying. These are my settings, not for them to be tampering with, but what can we do right. The date you're using, is that de password we need to fill in? My normal growatt password doesn't seem to work, but get no errors if i use today's date? Weird...

Solved using this method, now to block inbound traffic to the wifi dongle.

MSavisoft commented 1 month ago

I also had this same problem this morning at about 8am. Bloody annoying.

I can see from my PiHole logs that the only DSN lookup the DataLogger made since I got it all working with HA and Grott in March was a CNAME lookup this morning for server.growatt.com at 07:43 ... which is probably the cause of the problem.

NsLookup of server.growatt.com shows... Name: nlb-nbtf7vlspietfgd3ro.eu-central-1.nlb.aliyuncs.com Addresses: 47.254.130.145, 8.209.71.240 Aliases: server.growatt.com

Tomorrow I will try and decipher the info on this thread and the other and try and set it back. My best option may be to get PiHole to change the lookup.

daath commented 1 month ago

Mine did the same at around 7 am UTC (9 CEST) - was confused until later in the evening, when I decided to input the IP of the Grott instance. It gets data one time and then it stops working again?!

thoompje commented 1 month ago

I just created an A record in my DNS server.

A Record: server.growatt.com IP: x.x.x.x (Internal IP of Home Assistant / Grot).

Now it looks like its working again, lets wait for a few hours to see if it is really working. It's a really ugly solution because there will be no data in the shine phone app. But if this works it works.. Lets wait and see.

87Vortex87 commented 1 month ago

Update. Looks like the sensors agaim aren't working. I'll try to put the IP adres of HA in the dongle again and monitor. I hope they aren't resetting the dongle daily.

87Vortex87 commented 1 month ago

I just created an A record in my DNS server.

A Record: server.growatt.com IP: x.x.x.x (Internal IP of Home Assistant / Grot).

Now it looks like its working again, lets wait for a few hours to see if it is really working. It's a really ugly solution because there will be no data in the shine phone app. But if this works it works.. Lets wait and see.

This may be the solution if it turns out they reset the dongle daily.

87Vortex87 commented 1 month ago

Ok, update again: turns out i had to put in the IP adressen of HA again as endpoint in the wifi stick/dongle. Looks like they're sabotaging the configuration. It's called CONFIGURATION, I want control over that. This is the last Growatt product i ever bought.

87Vortex87 commented 1 month ago

Growatt does not want us to do these things, so no use in contacting them.. they'll do anything to keep us from doing this sh*t with me it is working by changing the server.growatt.com back to my grott ip address. but i did notice, when i take the power out of my shine link, the settings go back to growatt.com. thankfully i'm running a pihole dns server in my home so i guess i'll just make a static DNS entry in my dns server that makes server.growatt.com resolve to my internal grott ip address.

I have Pihole as well, I'll keep this solution in my back pocket if they decide to do this on a regular basis. I need the converter data in HA because my lights are controlled using the solar panel prodction as a sensor to determine is it's dark enough to turn on the lights in the house when presence is detected.

thoompje commented 1 month ago

I also tried to block incoming internet traffic to it and outgoing as well, but it is still resetting. I don't know if they pushed a script on it with some firmware or so which reset the device every few minutes. But the dirty solution I have now with the DNS manipulation is working fine, data is still getting into Home Assistant.

szemod commented 1 month ago

The dongle is not reset every day, but every 5 minutes! It can only be solved by rewriting DNA.

87Vortex87 commented 1 month ago

I've made a complaint at the Dutch ACM, which is a government agency which is involved with protection consumer right. Most likely nothing will happen, and if something happens it will take years, but better than nothing.

mjooris commented 1 month ago

it was working yesterday but it was only for a while. image the stupid thing does not let me change the dns server. it always goes back to 8.8.8.8 this is so frustrating. how did you do the dns manipulation if you cannot change the dns server to your internal dns?

87Vortex87 commented 1 month ago

The dongle is not reset every day, but every 5 minutes! It can only be solved by rewriting DNA.

Can you explain? what do you mean by rewriting DNA?

87Vortex87 commented 1 month ago

Growatt does not want us to do these things, so no use in contacting them.. they'll do anything to keep us from doing this sh*t with me it is working by changing the server.growatt.com back to my grott ip address. but i did notice, when i take the power out of my shine link, the settings go back to growatt.com. thankfully i'm running a pihole dns server in my home so i guess i'll just make a static DNS entry in my dns server that makes server.growatt.com resolve to my internal grott ip address.

I have Pihole as well, I'll keep this solution in my back pocket if they decide to do this on a regular basis. I need the converter data in HA because my lights are controlled using the solar panel prodction as a sensor to determine is it's dark enough to turn on the lights in the house when presence is detected.

Now that i'm thinking about it, we probably also need to block all incoming traffic to the wifi dongle, because if they push new settings where they decide their endpoint changes, from server.growatt.com to something else, that pihole rule won't work that long. So, probably:

  1. change setting of the wifi stick or intercept traffic and reroute using Pihole for example;
  2. block all incoming traffic to the IP-adress of the Wifi stick.
szemod commented 1 month ago

image Of course. The Domain Name Address is written back to server.growatt.com every 5 minutes, so it must be rewritten elsewhere. The DNS address of the dongle must be changed to an internal DNS server (Adguard in my case), where server.growatt.com must be redirected to the address that was previously set in the dongle.

mjooris commented 1 month ago

image Of course. The Domain Name Address is written back to server.growatt.com every 5 minutes, so it must be rewritten elsewhere. The DNS address of the dongle must be changed to an internal DNS server (Adguard in my case), where server.growatt.com must be redirected to the address that was previously set in the dongle.

can you change the dns server? i cannot. whenever i change the dns server, hit save. it always goes back to 8.8.8.8 :(

thoompje commented 1 month ago

You can rewrite 8.8.8.8 in your network pointing to your own dns server 😄 But that's more dirty then the solution before. I think my shine x dongle just use my own DNS server, because after changing A record on my own DNS server its working like a charm.

mjooris commented 1 month ago

yeah dont really want to do that... it will break other things.... i just found online documentation for a API. i did not know they had one. @muppet3000 https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf did you already take a look at this? i prefer to keep my data local but, if they have an api that works. is the second best thing i guess..

muppet3000 commented 1 month ago

Has anyone tried setting these values in the grott.ini file?

# To blocks commands from outside (to channge inverter and shine devices settings) specify blockcmd = True,
# specify noipf = True if you still want be able to dest ip addres from growatt server
# Specify noipf = True if you still want be able to dest ip addres from growatt server (advice only to use
# this for a short time)
#blockcmd = True
#noipf = True

I'm using grottserver so nothing is going to Growatt at all.

If you don't block incoming requests from growatt then regardless of what DNS etc. you're using they will still be able to push config to your dongle. The downside of blocking this though is that you also can't configure SoC & AC charge etc. from the website yourself, you'd have to temporarily change the setting to 'False' in order to allow pushing of configuration that you actually want.

thoompje commented 1 month ago

yeah dont really want to do that... it will break other things.... i just found online documentation for a API. i did not know they had one. @muppet3000 https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf did you already take a look at this? i prefer to keep my data local but, if they have an api that works. is the second best thing i guess..

I know you don't want that, but man what a sh*t hole is this with Growat.

dg362 commented 1 month ago

Like many, I need to maintain the Growatt server, plus the ability to use web interface (or App) to configure SoC or adjust the timing of Battery Charge, etc. This is because the tariff I am on needs to be managed when I'm charging my EV otherwise the battery will drain. Whilst I could consider grott-server that would lose the entire Growatt web and App interface and I'm not confident I can reproduce all the data points in HA the way I want.

My ShineWIFI-S reset this morning too. This was around 08:10 which is a little later than it was yesterday which was 07:45.

Growatt is making some changes as the web interface has had a few updates in terms of things like Battery now showing as Charging and Discharging on the main Energy Trend graph (but only on the Hour view, not the summarised Day/Month/Year views as that shows as Battery). They've also made the dialog page for the data logger bigger. I'm sure there are other updates as well.

I'm in the UK, and stating this in case the reset of the data logger is different between regions.

I have been trying to set up Notification automation in Home Assistant to notify me if the value of Solar Generation doesn't change for say 10 minutes. I've not yet gotten that to work because I'm confused between last_updated, last_changed, and last_reported as the states and how that affects the actual value of the sensor. This would at least allow a notification that between sunrise and sunset if the generation doesn't change the value for say 10 minutes then it is likely the data logger has been reset again.

87Vortex87 commented 1 month ago

Ok, yea, it has changed again after a few hours, they keep pushing the settings to the stick. This is becoming an ugly fight with Growatt. I think the only option is to block incoming traffic to the stick after changing the endpoint on the stick.

Edit: i've re-entered the IP i want into the stick and blocked all incoming traffic to the IP adres of the stick in my router. Let's see if this works.

Ryanmt2 commented 1 month ago

Not sure if its related but I am unable to access the web ui of my stick in any fashion now. not when it was connected to my wifi but I also hard reset it and when connecting via the hotspot I cant access http://192.168.10.100 either. Are you guys having any similar issues accessing the sticks config web ui?

87Vortex87 commented 1 month ago

Not sure if its related but I am unable to access the web ui of my stick in any fashion now. not when it was connected to my wifi but I also hard reset it and when connecting via the hotspot I cant access http://192.168.10.100 either. Are you guys having any similar issues accessing the sticks config web ui?

Yes, i have issues reaching the local web-ui of the stick as well, straight from the LAN to the stick. I can only push settings through server.growatt.com, where in the past i was able to reach the web-ui locally. But maybe i'm doing something wrong...

Ryanmt2 commented 1 month ago

I think they have perhaps pushed new firmware to disable the UI entirely. I have managed to re add my stick using the app so its not broken.

I have just done the same thing as you to block outbound traffic to the internet from the stick - I think it might fail on reboot as itll need to do a DNS lookup for server.growatt.com

daath commented 1 month ago

I changed the IP on the dataloggers, then enabled gblockcmd in my Grott HA UI and it's been working for the last 8 hours ...

thoompje commented 1 month ago

I changed the IP on the dataloggers, then enabled gblockcmd in my Grott HA UI and it's been working for the last 8 hours ...

I don't get this point, because if I change in the grot add-on the gblockcmd to enabled. What has this to do with the stick itself? Because they still can push it to the stick right?

So if I change the datalogger ip in the webinterface on growat.com and enabled the gblockcmd they can't change it anymore? Please some clarify on this, thanks!

dg362 commented 1 month ago

Firmware of my ShineWIFI-S is the same at 3.0.0.2 but that doesn't mean they haven't changed something in the backend of the web interface to ignore this setting. I am finding mine stops working every few minutes. Not sure if it is a number of 1 minute polls (I changed mine from 5 minutes to 1 minute) or that it resets at 5 minutes. What is somewhat weird is that in the Web Interface for the data logger "Set IP" this is remembering my value of 192.168.1.60 and is also remembering the password of 20240717 for today whenever I go into this part of the interface. But when you click Yes to apply, it just shows the word "Successful" in the bottom left of that dialog box rarher than the more normal "Successful" extra box that comes up when you send other settings elsewhere.

Yes, they are changing something. Whether this persists or not I guess we'll have to monitor. For now, this is completely useless as I cannot get my setting to stick for more than a few minutes at a time.

Ryanmt2 commented 1 month ago

So if I change the datalogger ip in the webinterface on growat.com and enabled the gblockcmd they can't change it anymore? Please some clarify on this, thanks!

The config push goes via grott so you can block it.

Ryanmt2 commented 1 month ago

Firmware of my ShineWIFI-S is the same at 3.0.0.2 but that doesn't mean they haven't changed something in the backend of the web interface to ignore this setting. I am finding mine stops working every few minutes. Not sure if it is a number of 1 minute polls (I changed mine from 5 minutes to 1 minute) or that it resets at 5 minutes. What is somewhat weird is that in the Web Interface for the data logger "Set IP" this is remembering my value of 192.168.1.60 and is also remembering the password of 20240717 for today whenever I go into this part of the interface. But when you click Yes to apply, it just shows the word "Successful" in the bottom left of that dialog box rarher than the more normal "Successful" extra box that comes up when you send other settings elsewhere.

Yes, they are changing something. Whether this persists or not I guess we'll have to monitor. For now, this is completely useless as I cannot get my setting to stick for more than a few minutes at a time.

Once you have it working go to the addon config, use the 3 little dots to edit in yaml and add gblockcmd: false this will stop growatt pushing updates.

Where are you changing the update interval? I cant see that option anymore and im at the default 5mins

thoompje commented 1 month ago

So if I change the datalogger ip in the webinterface on growat.com and enabled the gblockcmd they can't change it anymore? Please some clarify on this, thanks!

The config push goes via grott so you can block it.

Sure, is this not pushed via growat self? That's why it resets every time I thought...

87Vortex87 commented 1 month ago

I think you can only change the update interval in the local web-UI, but i can't reach that anymore. Can you guys reach the local web-ui of the stick?

87Vortex87 commented 1 month ago

Looks like i solved it as well, in a not so elegant way:

  1. update endpoints using server.growatt.com
  2. wait 20 seconds then block all incoming traffic to the stick using the router settings.

Bit forced, but seems to be working for now.

daath commented 1 month ago

I don't get this point, because if I change in the grot add-on the gblockcmd to enabled. What has this to do with the stick itself? Because they still can push it to the stick right?

Grott acts as a proxy, so all data from growatt goes through Grott first. This is why gblockcmd works. Datalogger never sees the command to reset ...

zutislo commented 1 month ago

Can someone elaborate more on this gblockcmd parameter. I have Docker Desktop running Grott (2.7.8) and mosquitto). Now, In HA i have Grott integration (1.1.0) but there is no option to edit yaml or anything like that. I opened grott.ini on docker image and set gblockcmd = false under Growatt section. THat it?

I've really had it with these Growatt i.... Instead of enabling charge/discharge controll when Load First mode they mess with stuff like this. Might as well just go on and install MODBUS reader that I have laying around somewhere. Untill they somehow block that too...

Ryanmt2 commented 1 month ago

There is also an alternative custom firmware for these dongles for those that are getting a bit sick of growatt.

https://github.com/OpenInverterGateway/OpenInverterGateway

dg362 commented 1 month ago

For those of us using Grott with HACS, I guess our current only solution is to edit the grott.ini file and set the two values below

# To blocks commands from outside (to channge inverter and shine devices settings) specify blockcmd = True,
# Specify noipf = True if you still want be able to set the destination ip addres from growatt server (advice 
# only to use this for a short time)
blockcmd = True
noipf = False

What I am not clear on is how often the grott.ini file is read. Does this happen only on HA restart, does this happen if the integration is reloaded, or is this read every time a new packet/message is received?

I initially set mine to True and True, waited for more than a minute as that is my poll interval, changed the IP in the data logger on the Growatt web interface, then set noipf=false.

But even with these settings, I can still go into the Growatt web interface and change the data logger "Set IP" value and it still says "Successful" which I would not expect given it should be blocking this now if I've understood these settings correctly.

zutislo commented 1 month ago

I have my ini set to this:

[Generic] invtype=sph blockcmd=true;

I just set the IP from shine app. This still reverts datalogger ip after cca 5 mins for me :/

muppet3000 commented 1 month ago

You have to restart grott for the ini file to be read again

zutislo commented 1 month ago

I have. And I have also posted too quick. It just seems that with this datalogger is not refreshing every minute. Last interval vas 4 minutes, then few updates every minute etc... oh well... Tx.

EDIT: Nope, datalogger was reset, blinking green and when accessed through hotspot mode in shine app... again server.growatt.com. On WiFi-X dongle.

dg362 commented 1 month ago

You have to restart grott for the ini file to be read again

I edited the grott.ini and set the two values to

blockcmd = True
noipf = true

I restarted HA. I waited 2 minutes to make sure this covered at least 1 poll from the data logger (mine is set to 1 per minute). In the Growatt web interface I changed the data logger to point to my local IP for Grott. I edited the grott.ini and set the two values to

blockcmd = True
noipf = false

I restarted HA.

I then went into the MQTT integration in HA, Configure, and then set topic to energy/growatt and started listening. I monitored this for 15 minutes, making sure that there was a message received once per minute with a new timestamp with new values. All were correctly received.

After those 15 minutes of monitoring, I went back into the Growatt web interface, logged out and logged back in again so it was a clean session with Growatt.

I then tried to change the data logger IP address, and this was allowed and showed 'Successful' in the bottom left of the dialog screen just the same as before. So I see no difference in behaviour on the Growatt web interface site with noipf = false which isn't what I expected as that suggests that Growatt can still talk to the data logger. Or is it the case that this is Grott talking to the data logger now? I can also still go into the inverter settings and change say the schedule or timing of AC Battery or whatever.

EDIT After a total of 21 minutes (messages via MQTT) this stopped working again without me changing anything. So the data logger must have 'lost' the IP address I change it to again despite the settings in place via grott.ini when using HACS. Actually not sure that is true as a new set of monitoring MQTT messages shows it is still okay.

dg362 commented 1 month ago

Where are you changing the update interval? I cant see that option anymore and im at the default 5mins

I did this via temporarily running grottserver as that is the only way I know how to change the polling interval on the dongle from the default of 5 mins to something else.

zutislo commented 1 month ago

I apologize for my stupid user syndrome... it is working, it's jut that in my grott.ini config i left semicolon after the blockcmd = True command. Debugged it and it seems to be blocking them chinese fellas. I hope this will stick now.

Thanks.

87Vortex87 commented 1 month ago

Looks like i solved it as well, in a not so elegant way:

1. update endpoints using server.growatt.com

2. wait 20 seconds then block all incoming traffic to the stick using the router settings.

Bit forced, but seems to be working for now.

For some reason this was not working, maybe outbound initiated traffic from the dongle allowed the reset to come back in the network, idk. Trying the suggested solution with blockcmd and noipf now, hope that will work.