Growatt_server integration not able to pull data from servers

muppet3000 commented 2 years ago

The problem

The issue described (and fixed) here: #80950 has re-appeared as a result of changes/blocking on the Growatt Server end. A new fix is required in the upstream library to fix this. In the meantime the workaround outlined in #80950 can be used while this is fixed.

What version of Home Assistant Core has the issue?

2022.11.0

What was the last working version of Home Assistant Core?

N/A - Will affect all versions

What type of installation are you running?

Home Assistant Container

Integration causing the issue

Growatt_Server

Link to integration documentation on our website

https://www.home-assistant.io/integrations/growatt_server/

Diagnostics information

See #80950

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 2 years ago

Hey there @indykoning, @jasperplant, mind taking a look at this issue as it has been labeled with an integration (growatt_server) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `growatt_server` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign growatt_server` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

growatt_server documentation growatt_server source _{^{(message by IssueLinks)}}

muppet3000 commented 2 years ago

If other people are experiencing this issue can they please let me know on here?

SACU-Student commented 2 years ago

Mine is no longer updating...last update 2 hours ago.

Just for info, there was a very recent update to the shinephone android app...now version 7.1.0. (maybe they tweaked the user agent)

muppet3000 commented 2 years ago

Mine is no longer updating...last update 2 hours ago.

Just for info, there was a very recent update to the shinephone android app...now version 7.1.0. (maybe they tweaked the user agent)

Thanks - I'll take a look at the network traffic from the new version of the app to see if there's anything special happening there. In the meantime I'll submit a short-term fix/patch for the next hotfix release of HA that will get it working again (until they block it again). Follow the hotfix/hack info on the other ticket to change the User-Agent field to something random again to make it work in the meantime.

mjooris commented 2 years ago

yup same issue here. i am still running with the workaround from last time but it stopped 2 hours ago again :(

muppet3000 commented 2 years ago

yup same issue here. i am still running with the workaround from last time but it stopped 2 hours ago again :(

All you need to do to make it work again is to change the 'User-Agent' string which has something like Linux Dalvic.... in it to any other string and it will start working again, but there's no guarantee for how long until they decide to block it again.

mjooris commented 2 years ago

indeed, got it working with: Mozilla/5.0 (X11; CrOS x86_64 8172.45.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.64 Safari/537.36

thx!

bigfloor commented 2 years ago

We should reverse engineer the Shine phone app to see wat they are using there. Will try to SSL decrypt the traffic from it and do a packet capture....

muppet3000 commented 2 years ago

We should reverse engineer the Shine phone app to see wat they are using there. Will try to SSL decrypt the traffic from it and do a packet capture....

There's no need, I already know what it's using - it's using the information from your local device. I have a plan on how to fix/resolve it, but I need to actually implement it.

In the meantime people will just have to use the same hack from last time.

muppet3000 commented 2 years ago

indeed, got it working with: Mozilla/5.0 (X11; CrOS x86_64 8172.45.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.64 Safari/537.36

thx!

Remember that the application is hitting the Android endpoint, so this will almost definitely get blocked at some point because it's pretending to be a Web Browser

timbba80 commented 2 years ago

I'm stuck with the workaround mentioned in the initial post. I have HA running in Raspberry 4 (Home Assistant OS, no docker / virtual environment) and growatt integration is in use. When I go to terminal and type "pip show growattServer", I'm getting an response "package(s) not found". Also when searching for files named as "initpy", none of them refers to growattServer.

Any ideas how to proceed?

bigfloor commented 2 years ago

I activated SSL Decryption on my FW, that works without breaking the Shine app (which is actually bad, as they therefore do not do any certificate pinning). And weird that they are using the info from the local device, cause them the integration should work if we mimic an iPhone for example. Really curious to see what the underlying issue is PS: Really thinking about setting up an extra RPi with a RS-485 to TTL converter, hook that up to my Growatt inverter and have that send info back to HomeAssistant, no more dependencies

bigfloor commented 2 years ago

I'm stuck with the workaround mentioned in the initial post. I have HA running in Raspberry 4 (Home Assistant OS, no docker / virtual environment) and growatt integration is in use. When I go to terminal and type "pip show growattServer", I'm getting an response "package(s) not found". Also when searching for files named as "initpy", none of them refers to growattServer.

Any ideas how to proceed?

Do this first: docker exec -it homeassistant bash

Works for my HA running on ESXi, with the 'SSH & Web Terminal' add-on installed

SACU-Student commented 2 years ago

thanks bigfloor !

I'm stuck with the workaround mentioned in the initial post. I have HA running in Raspberry 4 (Home Assistant OS, no docker / virtual environment) and growatt integration is in use. When I go to terminal and type "pip show growattServer", I'm getting an response "package(s) not found". Also when searching for files named as "initpy", none of them refers to growattServer. Any ideas how to proceed?

Do this first: docker exec -it homeassistant bash

Works for my HA running on ESXi, with the 'SSH & Web Terminal' add-on installed

That worked for me - I added the user agent from my android phone:

Mozilla/5.0 (Linux; Android 13; Pixel 6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Mobile Safari/537.36

and all is working again!

PMizz13 commented 2 years ago

Same problem here since about 6 hours ago. I'm struggling to figure out how to find and edit the file for the work around. Running HA OS on Pi4 so I'll just wait and cross my fingers for a fix again. >_<

timbba80 commented 2 years ago

@bigfloor thank you, this helped me as well!

Generally; freaking annoying that Growatt is acting in this way. Not sure why they do this (is there any rationale on this). To me as an basic user, who wants to have centralized information repostitory (Home Assistant), these changes do cause additional effort and harm. And if there is a provider who sees centralized solutions (such as home assistant) as an negative solution and wants people to use their own ecosystem instead....those organizations are not living in 2022. If I've had known this when making the deal for solar panels + inverter, I would had turned down the offer.

muppet3000 commented 2 years ago

Hi everyone, going to try and respond to everyone in one mega post, rather than quoting everyone's messages I'm just going to tag the people.

Firstly, I know what the issues is - it's the same as last time, they've just blocked the new User-Agent value. My intention is to submit a short-term fix/hack/workaround via a PR to the HA codebase this afternoon so that everyone can get their systems up and running again at least for the weekend.

Secondly, I have a longer term plan as to how I can fix this, it will just take time for me to implement i.e. I can't do it in the background while I'm at work today.

Thirdly, I have personal commitments all this weekend (it's my Daughter's birthday) so I will be lynched if I spend all my time writing code, therefore I will be slow to respond and there won't be a 'proper' fix until some time next week, so please be patient, especially since there will be a Dev build that will work around the problem.

Now onto all of the individual posts that require a response: @bigfloor - Thank you for your input, the user agent is specified by the phone, it's what's used to identify it on the server end, it 100% makes sense to use the Phone's ID for this. If you're interested in the solution I'm planning on implementing, please contact me directly via email and I'll kick off a chain when I have a moment. (This offer extends to anyone who's interested in my proposed implementation). RE: Alternative implementations for this - There's lots of ideas for this over on the forums here, feel free to take a look as there already lots of alternative options if that's the way you want to go: https://community.home-assistant.io/t/anyone-experience-with-connecting-a-growatt-solar-inverter/60430/257

@SACU-Student - Again, this is spoofing a mobile web-browser, not an android app, slightly different but will get it working at least in the short term

@PMizz13 - I'll update as soon as I've had my PR merged with a work-around

@timbba80 - It's unlikely that they're wanting to stop people using HomeAssistant altogether, it's more likely that they're seeing an increased load on their servers on account of everyone's HomeAssistant servers polling them for info every minute. That volume of requests soon adds up. Therefore, what (at least I think) they've done is implement some checking to drop anything not coming from their Android/iPhone app by inspecting the header field. The CORRECT solution (if I were them) would be to increase the processing power on their backend to keep up with the load.

NOTE - I will not close this ticket until the 'proper' solution is implemented, the hotfix/hack/workaround is just temporary.

timbba80 commented 2 years ago

Same problem here since about 6 hours ago. I'm struggling to figure out how to find and edit the file for the work around. Running HA OS on Pi4 so I'll just wait and cross my fingers for a fix again. >_<

If not installed already, install SSH terminal (settings -> add-ons -> add on store -> search with "SSH" keyword. You will find two options, "terminal & ssh" and "ssh & web terminal". I'm using "ssh & web terminal".
Once installed, start the SSH terminal and open it
In the SSH command prompt, type docker exec -it homeassistant bash
Type pip show growattServer. Take note of the "location"
type cd location* *replace location with the actual location from step 4
type cd growattServer
type vi __init__.py
go to the row which states headers = .......... It should be row nr. 30 (you can see current row at the bottom of the screen in vi
press i button on your keyboard (this enables edit mode in vi)
replace the current content after 'User-Agent':) as an example with the content mentioned above. As an example: headers = {'User-Agent': 'Mozilla/5.0 (X11; CrOS x86_64 8172.45.0)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Mobile Safari/537.36'} NOTE! As mentioned by @muppet3000, this might be only a short term fix as it's spoofing a web-browser instead of android app. So might be that they block this as well at some point.
Once you've done, press esc button (disables edit mode in vi)
type :x (saves the changes and quits vi)
type exit
type exit
reboot home assistant

If you want to quit VI without saving the changes (as an example if you did make an mistake and want to start from beginning), after pressing esc at step 11, type :q! (will close vi without saving the changes).

dannytsang commented 2 years ago

Minor changes to @timbba80 steps (I hope you don't mind) and thank you for writing up the step by step.

Changed step 7. to include the underscores in the file name and a shortcut in vi for :wq is :x (yes I'm that lazy 😉 ).

Same problem here since about 6 hours ago. I'm struggling to figure out how to find and edit the file for the work around. Running HA OS on Pi4 so I'll just wait and cross my fingers for a fix again. >_<

1. If not installed already, install SSH terminal (settings -> add-ons -> add on store -> search with "SSH" keyword. You will find two options, "terminal & ssh" and "ssh & web terminal". I'm using "ssh & web terminal".

2. Once installed, start the SSH terminal and open it

3. In the SSH command prompt, type "docker exec -it homeassistant bash"

4. Type "pip show growattServer". Take note of the "location"

5. type "cd location*" *replace location with the actual location from row 4

6. type "cd growattServer"

type vi __init__.py

8. go to the row which states "headers = .........". It should be row nr. 30 (you can see current row at the bottom of the screen in vi

9. press "i" button on your keyboard (this enables edit mode in vi)

10. replace the current content after 'User-Agent':) as an example with the content mentioned above.  As an example:
    `headers` = {'User-Agent': 'Mozilla/5.0 (X11; CrOS x86_64 8172.45.0)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Mobile Safari/537.36'} NOTE! As mentioned by @muppet3000, this might be only a short term fix as it's spoofing a web-browser instead of android app. So might be that they block this as well at some point.

11. Once you've done, press esc button (disables edit mode in vi)

type :x (saves the changes and quits vi)

13. type exit

14. type exit

15. reboot home assistant
If you want to quit VI without saving the changes (as an example if you did make an mistake and want to start from beginning), after pressing esc at step 11, type :q! (will close vi without saving the changes).

timbba80 commented 2 years ago

@dannytsang good catch! I'm quite a newbie when it comes to github, and didn't realize that adding two underscores makes them disappear.. I've modified the original content now with your inputs.

dsl10007 commented 2 years ago

A suggestion would be to make a list of 50 different strings and just randomly select one, every time it polls for data

muppet3000 commented 2 years ago

So it doesn't look like I'm going to be allowed to merge the fix that I was proposing for this, I have to merge it into the upstream library and then merge that in, which will take some time as I don't own the release process for the upstream library.

You have the workaround though, that's all I can do for now. As soon as there's an upstream release though I'll get the PR updated for here and in the meantime I'll keep thinking about the best way to implement a PROPER fix.

Enjoy your weekends!

DutchKillerbee commented 2 years ago

Thanks @muppet3000 for all your work! Fix works for me.

dsl10007 commented 2 years ago

Growatt servers are really poor again, even when accessing it from my app.

"Failed to connect to server" and "Network error" most of the time, even though there is no network problems, and it is just their crappy servers crapping themselves.

It seems that growatt is either unable or unwilling to make their servers work problem free.

stephenmhall commented 2 years ago

I can't get the Android app to connect, occasionally shows my address but not often, however the web based dashboard is working fine, maybe they have managed to block any Android based connection.

muppet3000 commented 2 years ago

Yeah they're really hit and miss again today, I think there's a risk we're associating any blip in connectivity with them trying to block something, remember, they're just pretty flakey! They can't go and block all android connections it would render their app pointless! Even if the Growatt website is more reliable if you look at the data that's there you'll find that when they've had a 'blip' like this all the data is missing for the duration of the blip meaning it's a problem on the server end rather than anything to do with them blocking stuff. The system receiving the data is offline during those sorts of windows not just the system we use to view it.

caddyblack commented 1 year ago

Please check your connection again. I could connect today after a few days no connection possible. Looks like Growatt server had problems.

muppet3000 commented 1 year ago

Hi All, I've submitted another PR with the library bump from the upstream library, I've also increased the timeout on the polling of the servers since the default setting for the wifi sticks to update is actually every 5 minutes, so polling every 1 minute was pointless. This ticket however will remain open until I can implement a better fix later this week.

prolife86 commented 1 year ago

Love that Home Assistant has API support for the very unreliable Growatt Servers. But there have been almost more outages than working days. By the time the solution has been implemented (and I applaud you, it is a though job!) Growatt is throwing another wrench in plan.

Might something like Grott be a solution?

chiefmaster81 commented 1 year ago

with Home Assistant 2022.11.3 and your fix it works again ! Thank you !

prolife86 commented 1 year ago

with Home Assistant 2022.11.3 and your fix it works again ! Thank you !

Have you tried reload and looking if it still works?

I pushed reload, and right away it broke again:

Unable to fetch data from Growatt server
Logger: homeassistant.components.sensor Source: components/growatt_server/sensor.py:43 Integration: Sensor (documentation, issues) First occurred: 10:13:37 (3 occurrences) Last logged: 10:14:15

Error while setting up growatt_server platform for sensor Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 281, in _async_setup_platform await asyncio.shield(task) File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 86, in async_setup_entry devices, plant_id = await hass.async_add_executor_job(get_device_list, api, config) File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 43, in get_device_list login_response = api.login(config[CONF_USERNAME], config[CONF_PASSWORD]) File "/usr/local/lib/python3.10/site-packages/growattServer/init.py", line 121, in login data = json.loads(response.content.decode('utf-8'))['back'] File "/usr/local/lib/python3.10/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

chiefmaster81 commented 1 year ago

reloaded and still working 2 minutes ago

prolife86 commented 1 year ago

Here is started working again after an hour. Probably were the Growatt servers acting up again...

dsl10007 commented 1 year ago

I just gave up.

Bought a Raspberry Pi Zero 2, unplugged the Growatt wifi dongle and connected it to the Raspberry Pi instead.

Installed Solar Assistant, set it up, and now i have data directly from the inverter, not via Growatt's incredibly laggy and crappy server, and i do not have to jump through hoops every once in a while, when they actively try to block us.

This is by NO means critizism of Muppet3000, you do amazing work, but between the laggy server and GW's constant attempts to shut us down, and with only 5 minute update intervals, the choice was easy.

I hope that me leaving, will let the servers have a bit more free clockcycles for you guys.

If someone is in the market for a Wifi dongle for an SPH inverter, i have one for sale :)

bartjanisse commented 1 year ago

@dsl10007 Hi,

I am quite interested in what you wrote above. Could you please elaborate on what is needed with this setup an how it is connected to the GW inverter.

Thanks in advance

muppet3000 commented 1 year ago

I just gave up.

Bought a Raspberry Pi Zero 2, unplugged the Growatt wifi dongle and connected it to the Raspberry Pi instead.

Installed Solar Assistant, set it up, and now i have data directly from the inverter, not via Growatt's incredibly laggy and crappy server, and i do not have to jump through hoops every once in a while, when they actively try to block us.

This is by NO means critizism of Muppet3000, you do amazing work, but between the laggy server and GW's constant attempts to shut us down, and with only 5 minute update intervals, the choice was easy.

I hope that me leaving, will let the servers have a bit more free clockcycles for you guys.

If someone is in the market for a Wifi dongle for an SPH inverter, i have one for sale :)

No hard feelings, thanks for the support completely understand the situation, I'm contemplating the grott route myself in the future anyway.

muppet3000 commented 1 year ago

@dsl10007 Hi,

I am quite interested in what you wrote above. Could you please elaborate on what is needed with this setup an how it is connected to the GW inverter.

Thanks in advance

Please can we not have this conversation here, the admins really don't like it when issues go off-topic. Please use the forums, that's what they're there for, I've posted links to it in this thread before. Even better if you want to start a new conversation over there and then post the link to it here.

This thread needs to be kept clear for discussion of the actual issue and it's resolution, not alternatives

Update - Here's a link to the mega-thread for all things growatt (where there has been lots of discussions on alternatives) - https://community.home-assistant.io/t/anyone-experience-with-connecting-a-growatt-solar-inverter/60430/257

muppet3000 commented 1 year ago

An update for everyone here, as you've hopefully seen there is a workaround in the latest release (2022.11.3) which is essentially just a bump in the library to get the user-agent value changed. Separately I am discussing with Indy (who maintains the python library that underpins this integration) the best long-term fix for this. My plans for the HA integration around this specific problem is to add 2 features:

The ability for users to specify the polling rate (the majority of growatt installations update data every 5 minutes so it's pointless querying any more often) - it will default to 5 minutes
The ability for users to provide their own 'User-Agent' value as part of the configuration of the integration - that way if Growatt ever break things again you can just change to something different. Note - This will be optional and if one isn't provided it will use whatever the underpinning library has

I will continue to support this library as long as I'm able and as long as Growatt still have an interface we're able to integrate against. I understand there are alternatives, I have no problems with people actively discussing alternatives, there are some great ones out there. I still feel there is a place for this integration though so I'm happy to continue supporting it. Please remember though that github is the place to track, update and support the bugs and issues associated with the various integrations, not to discuss their alternatives.

thoompje commented 1 year ago

Hi @muppet3000 is it not a option to generate random headers for each new session? So the growatt servers will never find all the same session headers?

muppet3000 commented 1 year ago

Hi @muppet3000 is it not a option to generate random headers for each new session? So the growatt servers will never find all the same session headers?

Yes, although I want to make them look as realistic as possible, which isn't the easiest thing (I've submitted a PR to the upstream library with some options and Indy & I are discussing options).

If we just generated a random string of 20 characters then it would actually be quite easy to block the generic case rather than blocking specific ones. I found a way of generating very specific Android looking user-agent headers so that's what I'd like to do. Importantly, whether that logic goes into the upstream library or this integration doesn't matter, what has to be supported though is the ability to specify the header in the library, which currently isn't a thing.

Chreutz commented 1 year ago

I'm seeing something similar to this today, last successful days pull was about nine hours ago.

Log from HA:

Logger: homeassistant.components.sensor
Source: components/growatt_server/sensor.py:43
Integration: Sensor (documentation, issues)
First occurred: 14:22:55 (1 occurrences)
Last logged: 14:22:55

Error while setting up growatt_server platform for sensor
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 281, in _async_setup_platform
    await asyncio.shield(task)
  File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 86, in async_setup_entry
    devices, plant_id = await hass.async_add_executor_job(get_device_list, api, config)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 43, in get_device_list
    login_response = api.login(config[CONF_USERNAME], config[CONF_PASSWORD])
  File "/usr/local/lib/python3.10/site-packages/growattServer/__init__.py", line 121, in login
    data = json.loads(response.content.decode('utf-8'))['back']
  File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

zanandreisfilippo commented 1 year ago

I add that now it doesn't even allow the installation of the integration. Is there any log I can post that might be helpful? Cattura

AdeZwart commented 1 year ago

I'm not sure if this is related, and please let me know if I need to open a separate issue for this:

My logs are flooded with errors. The stack traces are similar to the one @Chreutz posted:

2022-11-21 17:54:33.284 ERROR (SyncWorker_6) [homeassistant.components.growatt_server.sensor] Unable to fetch data from Growatt server
2022-11-21 18:04:33.221 ERROR (SyncWorker_3) [homeassistant.components.growatt_server.sensor] Unable to fetch data from Growatt server
2022-11-21 18:14:33.270 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.solar_panels_energy_today fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 527, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 722, in async_device_update
    raise exc
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 176, in update
    self.probe.update()
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 192, in wrapper
    result = method(*args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/components/growatt_server/sensor.py", line 197, in update
    self.api.login(self.username, self.password)
  File "/usr/local/lib/python3.10/site-packages/growattServer/__init__.py", line 121, in login
    data = json.loads(response.content.decode('utf-8'))['back']
  File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Typically, I see the error repeated for solar_panels_energy_today, solar_panels_lifetime_energy_output and solar_panels_lifetime_total_energy_input_1

Despite the errors, I do seem to have (up to date) data in Home Assistant. The log entries are just causing a lot of white noise when looking for other issues.

zanandreisfilippo commented 1 year ago

err.txt hello, I'm trying to install growatt cloud, I notice this error and the installation does not continue. (the error I linked earlier remains on the frontend). I had already installed your application and it worked very well. Then when I noticed it wasn't updating the data, I tried removing and reinstalling it, but it won't install anymore. I'll leave you the full errors I found in the full logs, I hope they help! PS: I'm using the latest version available and the VM is running via the official vmdk downloaded from the site and with esxi 5.5 virtualization.

Saentist commented 1 year ago

@muppet3000 is there any new change in API access?

muppet3000 commented 1 year ago

I've just run a couple of tests and I think they've blocked the User-Agent again, you can use the same hack that we've used before to change it to whatever you want. Tonight I've got some time set aside to implement the feature to allow people to change the UserAgent themselves. I also think that myself and Indy have agreed on how the upstream library will support this so that it can be generated randomly with each start of the plugin so if this happens again all you'd need to do is re-load.

In the meantime, just use the same hack we've used in the past (I didn't notice that the issue was happening again because I'm still using the old hack on my system) and I'll update later tonight with the status.

MartinHjelmare commented 1 year ago

Please note that Home Assistant won't accept addition of options to set the user agent or update interval for the integration.

Setting user agent is not something that the user should need to do. This setting belongs in the 3rd party library.

Every integration that supports config entries allows the user to turn off automatic polling and then the user can automate the update interval as needed with the entity service homeassistant.update_entity.

Side note: We don't recommend playing cat and mouse with service providers. Try to reach out to the service provider and see if you can find a way forward that is acceptable to both parties. If that's not possible, it may be better to move the integration to a custom integration.

muppet3000 commented 1 year ago

tries allows the user to turn off automatic polling and then the user can automate the update interval as needed with the entity service h

Thanks @MartinHjelmare - Especially for the note on the update_entity that's saved me some effort!

With regards to the updating of the user-agent, the plan is for this to be entirely handled in upstream library, with the option for a user to provide one if they wanted. However, if that won't get approved I'll just rely on the upstream library as suggested.

I have reached out to the service provider in the past but it just went to a black hole. However, it's on my list of things to try again tonight (which I now have more time for given that I don't need to implement the options to set the update interval :P)

Saentist commented 1 year ago

@muppet3000 try with web browser API Nice JSON answers, just need to be created template for requests. All in app values exist in web UI with even more values.

muppet3000 commented 1 year ago

@muppet3000 try with web browser API Nice JSON answers, just need to be created template for requests. All in app values exist in web UI with even more values.

I get that, but it would require a complete re-implementation of the library we use (unless there's one already) and an overhaul of this integration as well. Not a simple undertaking.

home-assistant / core