BUG - Growatt Server Connect & Stability Issues

[Saentist]

Again offline

Logger: homeassistant.components.sensor
Source: custom_components/growatt_server_api/sensor.py:44
Integration: Sensor (documentation, issues)
First occurred: 17:34:47 (1 occurrences)
Last logged: 17:34:47

Error while setting up growatt_server_api platform for sensor
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fa019425db0>: Failed to establish a new connection: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='server-api.growatt.com', port=443): Max retries exceeded with url: /newTwoLoginAPI.do (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa019425db0>: Failed to establish a new connection: [Errno -3] Try again'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 289, in _async_setup_platform
    await asyncio.shield(task)
  File "/config/custom_components/growatt_server_api/sensor.py", line 88, in async_setup_entry
    devices, plant_id = await hass.async_add_executor_job(get_device_list, api, config)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/growatt_server_api/sensor.py", line 44, in get_device_list
    login_response = api.login(config[CONF_USERNAME], config[CONF_PASSWORD])
  File "/usr/local/lib/python3.10/site-packages/growattServer/__init__.py", line 128, in login
    response = self.session.post(self.get_url('newTwoLoginAPI.do'), data={
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='server-api.growatt.com', port=443): Max retries exceeded with url: /newTwoLoginAPI.do (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa019425db0>: Failed to establish a new connection: [Errno -3] Try again'))

Is it again problem with UA And is it possible UA to be "HA_[HA-Ver]_Growatt-client-Version"

[muppet3000]

I haven't got the same issue and the integration is now written to choose a new user agent string every time Home Assistant starts, can you please try restarting home assistant and see if it fixes it?

Also, I've noticed sometimes that my home IP gets blocked, if you get a dynamic IP address from your ISP, try restarting your router to get a new IP address for your home and see if that fixes the issue.

If you still have an issue after that let me know and I'll try and investigate further.

[Saentist]

My IP is Dynamic but is always SAME. I installed latest Update of HA with restart HA.

Is there a way to see clear message "reason of error"?

[muppet3000]

The new version of the pypi library has better error handling so I'll try and include that in my next release. I'll also try and handle the errors better by catching them.

The error you're seeing is weird though, it doesn't seem like the same one you get when the UA gets blocked or when your IP gets banned.

Your dynamic IP doesn't seem very dynamic 😋. Does the mobile phone app still work from the same network? When my IP gets blocked then my phone stops working as well unless I switch it to 4/5G.

[Saentist]

if it help, log from in HA integration

Logger: homeassistant.components.sensor
Source: components/growatt_server/sensor.py:43
Integration: Sensor (documentation, issues)
First occurred: 17:34:44 (1 occurrences)
Last logged: 17:34:44

Error while setting up growatt_server platform for sensor
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fa01a446290>: Failed to establish a new connection: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='server-api.growatt.com', port=443): Max retries exceeded with url: /newTwoLoginAPI.do (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa01a446290>: Failed to establish a new connection: [Errno -3] Try again'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 289, in _async_setup_platform
    await asyncio.shield(task)
  File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 87, in async_setup_entry
    devices, plant_id = await hass.async_add_executor_job(get_device_list, api, config)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 43, in get_device_list
    login_response = api.login(config[CONF_USERNAME], config[CONF_PASSWORD])
  File "/usr/local/lib/python3.10/site-packages/growattServer/__init__.py", line 128, in login
    response = self.session.post(self.get_url('newTwoLoginAPI.do'), data={
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='server-api.growatt.com', port=443): Max retries exceeded with url: /newTwoLoginAPI.do (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa01a446290>: Failed to establish a new connection: [Errno -3] Try again'))

[muppet3000]

Is this still happening @Saentist ? I'm not sure if there's anything that I can do to fix it other than improve the logging. While this issue was happening did the ShinePhone app also stop working if it was on the same network?

[Saentist]

App was never stopped, integration after latest reload 30min ago reconnect.

But most strange is somehow bug https://github.com/muppet3000/homeassistant-growatt_server_api/issues/3 reappear in integrated in HA rename plant to alias, but this time without change in cloud.

Guess local CEE CDN is transfer finished ;)

[muppet3000]

That's really odd, on all fronts. The fact that nobody else logged a bug makes me thing it's specific to yourself. I just have no idea how to assist! Unfortunately I was away from my system yesterday otherwise I would have tried spinning up a new HA system using the credentials you gave me and see if it had the same problem.

Very weird. I'll leave this issue open as a reminder to improve the logging!

[muppet3000]

(Accidentally closed it - I've re-opened though.)

[GertJanArts]

Offline here too since 11:00 today

[muppet3000]

The API endpoints have been bouncing up and down all morning, this isn't the application getting blocked, I've been experiencing the same problem on the Android app over wi-fi and 4g all morning. I'll continue to monitor throughout the day.

Note: I plan to re-add the "server.growatt.com" URL to the endpoints to give people more options going forward if this continues to happen. It was only removed in the past because we thought it had been deprecated altogether.

[BobC76]

I'm just looking into my issue and saw this update. Sounds the same: for me as of today the integration can't connect but also looks they've been changing things at the server as I seem to be logged out of the growatt web dashboard frequently with a "The Server Has A 500 Error(2023-02-07 12:31:34)" message

I was using the core version, remembered this version would likely be ahead with any more recent fixes so tried to install it and it can't connect "Unknown error occurred". It was working fine yesterday.

[muppet3000]

I'm just looking into my issue and saw this update. Sounds the same: for me as of today the integration can't connect but also looks they've been changing things at the server as I seem to be logged out of the growatt web dashboard frequently with a "The Server Has A 500 Error(2023-02-07 12:31:34)" message

I was using the core version, remembered this version would likely be ahead with any more recent fixes so tried to install it and it can't connect "Unknown error occurred". It was working fine yesterday.

See my previous comment, this is certainly maintenance on the server side of things, once it stabilises things should start working again.

On the odd occasion I've been able to get to the web UI I've noticed big gaps in the data that has been uploaded indicating also that this is a server side problem.

[BobC76]

FWIW I locally changed the url to the "deprecated" server.growatt.com and seems to be working again. Obviously I don't know if that's coincidence and they stopped mucking around at the back end, but that url doesn't seem any less functional at any rate...

[muppet3000]

FWIW I locally changed the url to the "deprecated" server.growatt.com and seems to be working again. Obviously I don't know if that's coincidence and they stopped mucking around at the back end, but that url doesn't seem any less functional at any rate...

Correct, that URL is working (I've tested the same and intend to do a release with it back in tonight when I'm back from work), however I've looked at my data and it hasn't been updated in over an hour, so there's definitely still something happening on the back end.

In the ShinePhone app on my phone I've actually found that I'm now getting an "account locked" message which is new/different so I'll continue to monitor that. I'm also assuming that's related to the random changes they seem to be applying.

[Geo-Ron]

I was using the core version, remembered this version would likely be ahead with any more recent fixes so tried to install it and it can't connect "Unknown error occurred". It was working fine yesterday.

Haha, I did exactly the same thing today ... and ended up in the same place :-)

[muppet3000]

@BobC76 @Geo-Ron @GertJanArts - Can you guys all confirm that this has stabilised for you now?

I haven't had any problems for a few hours.

[catch56]

I had trouble this morning, and then got 'User is blocked' on the phone app from the same network which I've never seen before - although I only started using the HA integration 2-3 days ago so that's not saying much.

Shut down HA for about 6 hours, started it again, all OK since.

[muppet3000]

I had trouble this morning, and then got 'User is blocked' on the phone app from the same network which I've never seen before - although I only started using the HA integration 2-3 days ago so that's not saying much.

Shut down HA for about 6 hours, started it again, all OK since.

To get around the 'user is blocked' thing on my phone I created some new users via the Growatt website, including a new one explicitly for use by home-assistant. The FAQ on how to create more users is here: https://github.com/muppet3000/homeassistant-growatt_server_api/blob/dev/FAQ.md#how-do-i-share-my-credentials-with-you-for-testingadding-new-features-for-my-system-type (rather than sending me the credentials, just create more for yourself :P)

[catch56]

Thanks got that set up, felt a bit risky potentially sacrificing my one and only account! (although no such message on the website).

[BobC76]

@BobC76 @Geo-Ron @GertJanArts - Can you guys all confirm that this has stabilised for you now?

I haven't had any problems for a few hours.

Yep seems to have stabilised

[GertJanArts]

Stable here since 14:18u ShinePhone app and Growatt website were stable all day

[muppet3000]

I had trouble this morning, and then got 'User is blocked' on the phone app from the same network which I've never seen before - although I only started using the HA integration 2-3 days ago so that's not saying much.

Shut down HA for about 6 hours, started it again, all OK since.

My account that was blocked yesterday on my phone is now working again, so maybe try your previous account again. Maybe it was just a 12/24 hour block.

[muppet3000]

My accounts got blocked again - both on the server-api.growatt.com endpoint and the server.growatt.com endpoint. I was querying every 1 minute though. So I've changed that to every 3 minutes and I've created yet another account to test it with. It may be that they're just rate-limiting.

[Fannboy1337]

My account isnt blocked, but also all the sensors are not available. At the moment they are available, but they only update every 2 hours. Is it possible to get the information from the datalogger faster? Since a few updates ago, i also have 2 entities. One i think is the growatt and the other one is the datalogger that is connected with usb. ShinePhone is working too, but didnt show the current value.

[muppet3000]

My account isnt blocked, but also all the sensors are not available. At the moment they are available, but they only update every 2 hours. Is it possible to get the information from the datalogger faster? Since a few updates ago, i also have 2 entities. One i think is the growatt and the other one is the datalogger that is connected with usb. ShinePhone is working too, but didnt show the current value.

At present the issues with the server side of things makes this scenario very difficult to debug. Can I suggest that we wait a couple of days for all the server work to have stabilised and then you log a ticket with specifics of the issues your facing (if you're still facing them) and any functionality that you believe to be missing.

[TheWalrus2]

My accounts got blocked again - both on the server-api.growatt.com endpoint and the server.growatt.com endpoint. I was querying every 1 minute though. So I've changed that to every 3 minutes and I've created yet another account to test it with. It may be that they're just rate-limiting.

Do you agree now? We have major issue with the API... i going to change to every 3 minutes to

[BobC76]

At the moment they are available, but they only update every 2 hours.

Their server has definitely been flaky today:

Logger: custom_components.growatt_server_api.sensor
Source: custom_components/growatt_server_api/sensor.py:178
Integration: Growatt Server API (documentation, issues)
First occurred: 10:14:00 (7 occurrences)
Last logged: 12:04:08

Unable to fetch data from Growatt server

On their server dashboard page, I also see gaps of 20-30 mins in the log between those hours.

Another thing to check, and this may be nothing to do with people's issues, but have you eliminated the possibility of local network issues affecting the upload from your datalogger to their server? Is the data on their server normally updated every 5 mins, or whatever your logger update frequency is set to? Is it only the HA sensor that is updating every 2 hours?

My wifi dongle is a long way from the nearest wifi access point, so the datalogger signal reports "poor" and so aside from the integration not able to reach their server, if the dongle can't reach it either there would be gaps with no update.

[muppet3000]

At the moment they are available, but they only update every 2 hours.

Their server has definitely been flaky today:

Logger: custom_components.growatt_server_api.sensor
Source: custom_components/growatt_server_api/sensor.py:178
Integration: Growatt Server API (documentation, issues)
First occurred: 10:14:00 (7 occurrences)
Last logged: 12:04:08

Unable to fetch data from Growatt server

On their server dashboard page, I also see gaps of 20-30 mins in the log between those hours.

Another thing to check, and this may be nothing to do with people's issues, but have you eliminated the possibility of local network issues affecting the upload from your datalogger to their server? Is the data on their server normally updated every 5 mins, or whatever your logger update frequency is set to? Is it only the HA sensor that is updating every 2 hours?

My wifi dongle is a long way from the nearest wifi access point, so the datalogger signal reports "poor" and so aside from the integration not able to reach their server, if the dongle can't reach it either there would be gaps with no update.

No issues with data being uploaded here. It's all server problems, I'm certain of it (I have a shine-lan box hard-wired to my ethernet 30cm away from my RF Dongle). It has all just stabilised again now as the evening has set in here in the UK.

I have decreased my polling interval in HA to 3 minutes (I was polling every 1 minute and my account got blocked again). I'll update again tomorrow if this remains stable. We may just have to accept less frequent updates.

Separately I'm going to have a look at some of the more 'local' options e.g. Grott, but while the API calls are still (kind of) working I'm going to keep supporting this integration. At least until I have a really solid, easy-to-use alternative.

[muppet3000]

@mjooris & @JimGermany

Just tagging you over here as you mentioned the similar issues over in the HA Core Repo: https://github.com/home-assistant/core/issues/87696

If you guys are happy to "hack" things in HA Core, then I'd really recommend just switching over to this version of the integration instead. I'm actively maintaining things over here and I'm able to deploy rapid fixes to critical issues.

The 'dev' build of this repo has the ability to use the growatt.server.com endpoint again and I plan to do a formal release as soon as I've fixed one final bug.

Notably, there's actually nothing I can really do from a code perspective to resolve the issues that we've encountered with the Growatt servers over the last 2 days other than to encourage people not to poll for information any more than every 3 minutes (the default for this Integration is 5).

[edibg]

Hi guys, just to add - starting from 30min ago I have a similar issue with the login in server-us.growatt.com. I use growattServer api to host a local http server with summary of my solar data. I check it this morning around 06:30 utc time and it was showing the data correctly. Since then I got the login error. I try as requested created new user - but the error remains the same even with the new user. The Andoid app is working fine and showing statistics.

I guess Growatt changed something within login routines This is my code.

def get_solar_data():
      api = growattServer.GrowattApi()
      api.server_url = 'https://server-us.growatt.com/'
      #login_response = api.login('XXX', 'XXX') #app user and pass
      login_response = api.login('XXX', 'XXX')  #new user and pass
      print('Login response is:%s'%login_response)  #Login response is:{'msg': '507', 'success': False}

      #plant_list = api.plant_list(login_response['user']['id'])     *****#get error   File "c:\Stani\IntranetX\pv_data\server.py", line 87, in get_solar_data plant_list = api.plant_list(login_response['user']['id']) KeyError: 'user' *****

      plant_list = api.plant_list(api.login('XXX', 'XXX'))*******#get error   File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\growattServer\__init__.py", line 135, in plant_list raise RuntimeError("Request failed: %s", response) RuntimeError: ('Request failed: %s', <Response [302]>)**** 

[muppet3000]

Hi guys, just to add - starting from 30min ago I have a similar issue with the login in server-us.growatt.com. I use growattServer api to host a local http server with summary of my solar data. I check it this morning around 06:30 utc time and it was showing the data correctly. Since then I got the login error. I try as requested created new user - but the error remains the same even with the new user. The Andoid app is working fine and showing statistics.

I guess Growatt changed something within login routines This is my code.

def get_solar_data():
      api = growattServer.GrowattApi()
      api.server_url = 'https://server-us.growatt.com/'
      #login_response = api.login('XXX', 'XXX')   #app user and pass
      login_response = api.login('XXX', 'XXX')    #new user and pass
      print('Login response is:%s'%login_response)    #Login response is:{'msg': '507', 'success': False}

      #plant_list = api.plant_list(login_response['user']['id'])   *****#get error   File "c:\Stani\IntranetX\pv_data\server.py", line 87, in get_solar_data plant_list = api.plant_list(login_response['user']['id']) KeyError: 'user' *****

    plant_list = api.plant_list(api.login('XXX', 'XXX'))*******#get error   File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\growattServer\__init__.py", line 135, in plant_list raise RuntimeError("Request failed: %s", response) RuntimeError: ('Request failed: %s', <Response [302]>)**** 

Hi @edibg - Just a note, this chain is specifically focussed around the Home Assistant Integration for Growatt, however, the issue you're experiencing here is the same as what the Growatt Integration is suffering (because they both use the same library under the hood: this one, there is a specific bug against that library here), all that said, you're welcome to join the discussions here too.

I believe the 507 code is the new error code for "account locked", when you use the same account on your phone does it come up as locked? Your account getting locked could be related to how often you're polling for information, I've noticed that the more often I poll for information the quicker my account gets blocked.

I haven't noticed any changes to the login routines but I could be wrong.

[edibg]

I believe the 507 code is the new error code for "account locked", when you use the same account on your phone does it come up as locked? Your account getting locked could be related to how often you're polling for information, I've noticed that the more often I poll for information the quicker my account gets blocked.

Hi Muppet, until this morning I used same account for my home hosted API instance, work hosted API, my wife and I used same account on our android devices too whitout any issues. Currently our andoid apps keep working with the same credentials.

Now, as it was pointet out in the readme(until this morning I used few months old version of the API) will try to forward all my phone traffic trough my OpenVPN interface on the mikrotik so I can sniff what the app is talking to the server-us.growatt.com ( bay be grab the user id) but will see...I am not so good in those areas.

Noted - I will monitor other topic too.

Post next comment note : @muppet3000 I add you so you can access my solar data, but currently my setup is pointing towards server-us.growatt.com.

[muppet3000]

I believe the 507 code is the new error code for "account locked", when you use the same account on your phone does it come up as locked? Your account getting locked could be related to how often you're polling for information, I've noticed that the more often I poll for information the quicker my account gets blocked.

Hi Muppet, until this morning I used same account for my home hosted API instance, work hosted API, my wife and I used same account on our android devices too whitout any issues. Currently our andoid apps keep working with the same credentials.

Now, as it was pointet out in the readme(until this morning I used few months old version of the API) will try to forward all my phone traffic trough my OpenVPN interface on the mikrotik so I can sniff what the app is talking to the server-us.growatt.com ( bay be grab the user id) but will see...I am not so good in those areas.

Noted - I will monitor other topic too.

If you're willing to share a set of credentials with me - see here I'll see what I can do in the short term to unblock your issue with the login, but I can't promise what the long term situation will hold (see my next comment).

[muppet3000]

Hi Everyone,

This is a bit of a long update, but I'm trying to capture everything in one post so that I don't have to keep responding to things separately.

Current Situation

• As you've all seen, Growatt have been changing various things on their endpoints resulting in accounts getting blocked - mine included
• On my various test systems I have seen polling intervals all the way up to 5 minutes (the default for this integration) get blocked, I have one running every 10 minutes and that seems to be ok (but not very useful for updates)
• If your account gets blocked the short term solution is to go to the Growatt Website and create a new account for use with the integration, follow the FAQ here but keep the credentials for yourself instead of sending them to me.
• Whenever my account is getting blocked I'm creating a new one for HA and starting again, I'm waiting to find out whether ones that got blocked yesterday have been unblocked again.
• I cannot keep responding to every single message on this thread throughout the day, I have a job and constant engagement here is causing my work to suffer
• I'm at the point where the constant 'fight' with Growatt is exhausting and just a matter of time before it becomes pointless, I may turn my focus to other options (see more info below)

Debug Steps Here are some steps for people to debug if they're having issues: 1 - Test the same credentials on the ShinePhone App - if it comes up saying your account has been locked, then it has, it seems to only last about 24 hours, but the short-term fix is to create another set 2 - If your credentials work in the App then try restarting Home Assistant 3 - If Home Assistant is working but the data isn't updating then check the data on the App/Website - the last few days we've seen data only being uploaded intermittently while they've been changing with things - if there's no recent data in the website/app then there won't be recent data in HA - simple as that

Next steps This is the tricky one, I've always said that I will attempt to maintain this Integration and the underpinning library for as long as it makes sense to. While the underpinning library still has it's uses for occassional data pulling (i.e. every 10 minutes), it is clearly becoming less and less viable for instantaneous usage. The thing I loved about this Integration when I first came across it (before I became it's sole maintainer) was how simple it was to use/setup in HA, and I want to work to find an equally easy option that relies on local data rather than the Growatt servers. My plan is to fire up an instance of Grott and see what I can get running with HA, I've never tried it before and it'll take me a bit of time to get my head around. If it looks like a viable alternative then I'll work out a way forward to make this integration use local data instead. All I know is that I'm reluctant to take any further steps in the direction of "api calls" given that Growatt seem intent on blocking frequent access, who knows what they'll try next!

In the meantime, please just be mindful that I'm as much of a victim in this whole situation as everyone else, I don't know all the answers, I'm just muddling through like the rest of you.

Today I'm going to try my best NOT to respond to every comment that comes up about this issue, I've set some time aside tonight after work to catch up on it all and will respond then. Again, I'm sorry that we're all having this problem, but there is nothing I can do to resolve it.

[catch56]

I started looking at grott yesterday, haven't actually got it working yet, but this PR and the linked issues might help with the HA documentation which is quite outdated https://github.com/johanmeijer/grott/pull/280

[TheWalrus2]

Hi Everyone,

This is a bit of a long update, but I'm trying to capture everything in one post so that I don't have to keep responding to things separately.

................... Other API variants must have the same issues... I saw someone capture the data from the inverter directly but then you have to do some cringy hardware stuff. Soldering wires to pin 3 and 4 of the port etc... and connect it with an arduino or pi.

[BobC76]

I don't claim that this will address everyone's issues but FWIW, I set the URL as server.growatt.com, set up a dedicated HA 'browse' user, and haven't altered the default 5min polling. My HA log data appears to have had no sustained outages in the past 12 hours or so.

Clearly as @muppet3000 has said, not a lot can be done if the server itself is locking out accounts. I've seen it introduce a verification code logging in 'manually' when I tried my password manager auto scripting, so its clear they have some defensive code in place that makes this integration's use of their interface always likely to be be fragile. I see the same with other HA integrations that try to emulate a mobile client. Apart from exploring grott, I think the best thing everyone experiencing issues can do is lobby Growatt to support an API formally for 3rd party integrations.

[Geo-Ron]

@BobC76 @Geo-Ron @GertJanArts - Can you guys all confirm that this has stabilised for you now?

I haven't had any problems for a few hours.

I started receiving data again at 16:00 CET at the 7th of februari, everything stable since then :-)

And I've created an account for @muppet3000 on my plant. Have fun!

[TheWalrus2]

No it is a mess server side. They are really bullying the API users.

[vingerha]

Main and subaccounts work fine on app and website, HA integration do not connect, neither the 'old' core one nor the HACS one.

[muppet3000]

Hi everyone, here's the findings/summary from today:

1. My change to 4 minute polling seems to have been ok, but until it's been 24 hours (especially overnight when China comes back online) I'm not holding my breath

2. The server-us.growatt.com endpoint seems to be blocking in an entirely different way, I will attempt to discover how to work around it but I'm not sure what I can offer. (@vingerha - I've written this as a result of our discussions separately)

3. For anyone thinking this is related to the 'captcha' code that occasionally appears on the Growatt web-pages, it isn't this integration uses a library that has never had to use that method, so that is not the problem. This is simply just a case of the Growatt servers rate limiting, for a service that they don't officially provide (or even know about).

4. I had an outage where no information was uploaded to the Growatt servers for approximately 2 hours this afternoon, I made no changes and it started work again on its own, so I'm blaming that on the server and nothing to do with the integration

5. I plan to do a release tonight which contains the recent fixes for bugs I've been working on (nothing related to these issues, discussed here, but it's nice to tie up those loose ends)

6. After this release I'm going to stop work on any more bugs and features until there's some stability or clarity on what's happening. Instead I'm going to focus my attention on seeing how viable 'Grott' is as an "easy to use" alternative. It will take some time for me to do that, I'll remain active here in terms of chat etc. but I'm unlikely to go adding any major new features etc.

7. To a certain extent I can understand the logic from Growatt, they receive no additional money for the use of their website/app past the point of original installation of our Growatt equipment. It costs them to host these servers for the limited services they do provide, therefore they have no obligation to provide support for all the extra server calls that we're making to gather our data. On the flip side, wouldn't it be nice if they offered an API service! I'd even consider paying for it!!!

Once again, I'm as frustrated as all of you are that this is happening, especially after all the effort I went through in January to pull this Integration out of HA Core and into a custom component to better support it.

Finally like I said above, I'll continue to engage here and I'll certainly keep the Integration going if the 5-minute polling interval seems to stick as an "acceptable" rate. But it does feel like it's only a matter of time before the next thing comes along to upset usage! At a minimum though I'll keep it running until I can put some work together on how best to integrate something like Grott with HA, maybe even with another new Integration :)

[muppet3000]

Final one from me today. The release is out there: https://github.com/muppet3000/homeassistant-growatt_server_api/releases/tag/v1.0.2 The only real relevance to this thread is that it allows you to choose the growatt.server.com url when configuring the integration.

I'm signing off now, I'm tired and I'm not going to magically fix the growatt servers. Tomorrow (or over the weekend more likely) my experimentation with Grott begins!

[BobC76]

Thanks @muppet3000 for looking into it.

Just in case it's useful to others to know. As I said above, I had no issues all day, except occasional outages I think at server side. But then I restarted HA, for different reasons, and got 507 responses from the login steps in server.py and, when I tried reinstalling the integration, the initial setup code.

So I changed user account and magically no problem. I have no idea why the login request would give a 507 response, maybe that will be a clue for someone...

Otherwise @muppet3000 I look forward to hearing how you get on. It certainly looks like nailing jelly to the wall trying to guess how to work around an unsupported interface.

FWIW I see the point about the challenges supporting this from Growatt's perspective, and lack of a return. But I personally think in a (near) future when home energy management systems need to manage demand to local generation capacity, interoperability between devices from different vendors is likely to become a core feature, if not to Growatt maybe their competition.

[muppet3000]

Thanks @muppet3000 for looking into it.

Just in case it's useful to others to know. As I said above, I had no issues all day, except occasional outages I think at server side. But then I restarted HA, for different reasons, and got 507 responses from the login steps in server.py and, when I tried reinstalling the integration, the initial setup code.

So I changed user account and magically no problem. I have no idea why the login request would give a 507 response, maybe that will be a clue for someone...

Otherwise @muppet3000 I look forward to hearing how you get on. It certainly looks like nailing jelly to the wall trying to guess how to work around an unsupported interface.

FWIW I see the point about the challenges supporting this from Growatt's perspective, and lack of a return. But I personally think in a (near) future when home energy management systems need to manage demand to local generation capacity, interoperability between devices from different vendors is likely to become a core feature, if not to Growatt maybe their competition.

With regards to the issues when restarting I believe I started the same issue (I've just created another use and used that to with around it but that's besides the point). The reason it breaks when you restart is because it has been blocked at some point earlier but continues to work because the integration doesn't re-authenticate each time it polls for data, it uses a session cookie which doesn't get blocked. When you then try to re-authenticate as part of a restart or an update it can't.

Not a solution to the problem but an explanation at least.

[malosaa]

When i wrote my code back at hs4 (homeseer) some time ago, it still works for me retrieving the data. But its a old python script that retrieves everything i need from growatt. Never had a issue with it. Besides like now that they are having an issue, but it retrieves the data for me but with a 0 value of current usage. Total data is retrieved tho.

I know it has nothing to do with home assistant, but just want to say my very old way still works and never stopped working.

[muppet3000]

Just out of curiosity, are people still having issues with data not uploading to the growatt servers? I acknowledge that there are issues with accounts getting locked etc. but I'm still having huge gaps in the data that's getting pushed to Growatt. Over the last 4 hours I've only had about 5 data points pushed to the graphs.

[edibg]

Nothing missing on the US server. Maybe they are change something on the backend that causes the missing data/servers offline.

A few months ago(again on the us one) I had every 4 weeks one day without data available, may be the maintenance that they are doing now causes this missing data.

[muppet3000]

Yeah almost certainly likely, it's just decided to throw another data point up there, so they're clearly doing something. I remember something similar happening in the past for about a week and then it all started working again. This is one of the other advantages of the Grott approach, everything becomes local and therefore can be more reliable than the servers. Itching to find the time this weekend to get my hands on Grott!

[GertJanArts]

Uploading to the Growatt server has never be an issue. Never gaps here.

[vingerha]

@muppet3000 I got grott installed today, trying this out too .... driven by your view on questionable future of the growatt api service. It works and provides the data I need but I no longer have data in the growatt app nor the website. The documentation seems to allow a upload to server.growatt if you setup sniffing but that is too much for my not-so-powerful nuc. Personally, I prefer to have all but main goal is: continuiity in the feed to HA