search

sander1988 / Indego

Home Assistant Custom Component for Bosch Indego Lawn Mower
Apache License 2.0
92 stars 29 forks source link

Connection error bosch indego APi #204

Closed Tacchi1983 closed 3 months ago

Tacchi1983 commented 10 months ago

The integration became offline too after ha update. I tried to reinstall like u recommended but i cant connect it anymore. I can login but then i get a warning that connection with bosch indego API didnt succeed.

Log: Deze fout is ontstaan door een aangepaste integratie.

Logger: custom_components.indego.config_flow Source: custom_components/indego/config_flow.py:92 Integration: indego (documentation) First occurred: 11:24:49 (1 occurrences) Last logged: 11:24:49

Error while retrieving mower serial in account! Reason: 403, message='Forbidden', url=URL('https://api.indego-cloud.iot.bosch-si.com/api/v1/alms')

michaelmog88 commented 10 months ago

Same here

jm-73 commented 10 months ago

Please write: Version of HA Version of the integration When did work correctly last time? (Date and version used then) When did it stop working?

Tacchi1983 commented 10 months ago

It stopped working after latest ha version 2023.11.0 updated at 2nd of november. It worked nicely before. Integration version 5.6.1, the same as before when it still was working. Last working ha version 2023.10.5

TanteFinn commented 10 months ago

Hi there, I just wanted to confirm 403 issue after updating to HA from version 2023.10.5 to version 2023.11.1.

LJvHeck commented 10 months ago

Hi, Also got the error "failed with HTTP status code: 403" after updating HA to 2023.11.1. Integration version 5.6.1

Tried to reinstall, but that didn't work.

Blackmood667 commented 10 months ago

Same here

AcciTo commented 10 months ago

And here

matilbianco commented 10 months ago

Also here : Integration stopped working once upgraded HA from 10.4 to 11.0. Removed then add HA integration failed with error message : "The connection to the Bosch Indego API failed!" Also tried again after re-installed Chrome extension, same error. From HA Logs : 

"Logger: custom_components.indego.config_flow
Source: custom_components/indego/config_flow.py:92
Integration: Bosch Indego Mower ([documentation](https://github.com/jm-73/Indego))
First occurred: 22:27:59 (3 occurrences)
Last logged: 22:31:37"

Error while retrieving mower serial in account! Reason: 403, message='Forbidden', url=URL('https://api.indego-cloud.iot.bosch-si.com/api/v1/alms')
PiotrasHASS commented 10 months ago

Same here

Core 2023.11.1 Supervisor 2023.10.1 Operating System 11.1

tobaco commented 10 months ago

Stopped working for me too, but did not yet update to 2023.11.x

Home Assistant 2023.10.5
Supervisor 2023.10.1
Operating System 11.1
jlaur commented 10 months ago

It stopped working for me too. In openHAB. 😄 Bosch changed something on their side, so it's highly unlikely that it's a regression in the integration.

Feldermann commented 10 months ago

I got the following error2:

Logger: pyIndego.indego_base_client Source: custom_components/indego/config_flow.py:85 Integration: Bosch Indego Mower (documentation) First occurred: 08:54:42 (2 occurrences) Last logged: 08:55:39

Request to 'https://api.indego-cloud.iot.bosch-si.com/api/v1/alms' failed with HTTP status code: 403

and

Logger: custom_components.indego.config_flow Source: custom_components/indego/config_flow.py:92 Integration: Bosch Indego Mower (documentation) First occurred: 08:54:42 (2 occurrences) Last logged: 08:55:39

Error while retrieving mower serial in account! Reason: 403, message='Forbidden', url=URL('https://api.indego-cloud.iot.bosch-si.com/api/v1/alms')

Core 2023.11.1 Supervisor 2023.10.1 Operating System 11.1 Frontend 20231030.1

35aportal commented 10 months ago

Also the same for me. After a restart for some HACS updates, the Indego integration now returns 'unavailable' for all sensors. I tried to remove and reinstall, but see "The connection to the Bosch Indego API failed!"

Core 2023.11.0 Supervisor 2023.10.1 Operating System 11.0 Frontend 20231030.1

Larnak23 commented 10 months ago

Same for me

jipijajay commented 10 months ago

Also the same for me. After a restart for some HACS updates, the Indego integration now returns 'unavailable' for all sensors. I tried to remove and reinstall, but see "The connection to the Bosch Indego API failed!"

Core 2023.11.0 Supervisor 2023.10.1 Operating System 11.0 Frontend 20231030.1

same here

sander1988 commented 10 months ago

Same here. Nothing changed in my setup. So I think that something has changed on the Bosch API side. I will debug.

sander1988 commented 10 months ago

The Azure WAF (the Bosch API runs on Azure) was blocking our user-agent for unknown reasons. Similar issue as #119.

I have changed the user-agent to be unique for each HA instance. Now the 403 issue is gone. I have 2 PR containing the changes.

sander1988 commented 10 months ago

@jm-73 - Please wait with merging. I see some 403 errors again over the last few hours. I expect to have some time to debug this next weekend.

Larnak23 commented 10 months ago

Thanks in advance

sipossz commented 10 months ago

I'm a little bit afraiding about similar vendor bans will come for reverse engineered unpublished api usages :( https://www.home-assistant.io/blog/2023/11/06/removal-of-myq-integration/

sander1988 commented 10 months ago

I'm a little bit afraiding about similar vendor bans will come for reverse engineered unpublished api usages :(

https://www.home-assistant.io/blog/2023/11/06/removal-of-myq-integration/

Possible, but I don't think that's case here...

sid3windr commented 10 months ago

I've applied the patch and still got a 403. When I replaced "HomeAssistant" by "Test" in the user agent string, adding my mower worked. Might be a coincidence, might very well not be...

Larnak23 commented 10 months ago

Hi any news

sander1988 commented 10 months ago

Hi any news

At this point I'm look for a way to create a fix, but I don't want to just change the user agent again and see it blocked once more within a couple of days.

Maybe let the user choose the user agent during integration setup? Maybe a bit complex for non-technical users, but the best I can think of.

Any other ideas?

NissarAK84 commented 10 months ago

I am getting this error message:

Screenshot 2023-11-20 at 00 30 39

Is it related to this issue?

sid3windr commented 9 months ago

I am getting 403s now, even after my user agent change to Test/Indego 3 days ago.

I did not have any issues with the OpenHAB binding for Indego, is the HA integration more agressive requesting status updates? (OH version has lower refresh, then when the mower is detected to be mowing, it has a different refresh, for example).

sander1988 commented 9 months ago

is the HA integration more agressive requesting status updates?

I don't think so. I have checked the code, this is what I see:

= 865 requests a day when the mower is idle. Not that much.

And I don't think this is the issue. You would a expect the API to return status code 429 instead of a 403 when this is the issue.

sander1988 commented 9 months ago

I am getting this error message:

Screenshot 2023-11-20 at 00 30 39

Is it related to this issue?

It might. Check the logs for a 403 error to be sure (see description of this issue for an example).

sid3windr commented 9 months ago

And I don't think this is the issue. You would a expect the API to return status code 429 instead of a 403 when this is the issue.

You would. And I've seen 429 when using OpenHAB with more aggressive timing + the Android app at the same time. But something is tripping up the WAF or Bosch themselves leading to these blocks.... (Indeed the above timings look perfectly normal to me..)

35aportal commented 9 months ago

The Azure WAF (the Bosch API runs on Azure) was blocking our user-agent for unknown reasons. Similar issue as #119.

I have changed the user-agent to be unique for each HA instance. Now the 403 issue is gone. I have 2 PR containing the changes.

Sorry for what might be a basic question, but how do I try the patch that you suggested? I have installed the Indego integration via HACS and see the 'Indego' repository files on my HA machine, but not the pyIndego files in which you made changes.

Thanks in advance.

jm-73 commented 9 months ago

As per 2023-11-24 the integration seems to be working again. Apparently Bosch have made som changes to their servers again. This time to our advantage.

sid3windr commented 9 months ago

Cannot confirm:

2023-11-25 00:00:56.565 ERROR (MainThread) [pyIndego.indego_base_client] Request to 'https://api.indego-cloud.iot.bosch-si.com/api/v1/alms/xxxxxxxxxx failed with HTTP status code: 403

This is both with Test/Indego and HomeAssistant/Indego as user agents.

NissarAK84 commented 9 months ago

I am getting this error message:

Screenshot 2023-11-20 at 00 30 39

Is it related to this issue?

I concur that the issue is resolved (for now at least) as mentioned by @jm-73 .

lajosdani commented 9 months ago

it is working for me as well, without any changes made with the integration.

35aportal commented 9 months ago

Yes, me too - no changes, just a reinstall and it works OK

jipijajay commented 9 months ago

Yes, me too - no changes, just a reinstall and it works OK

That also worked for me. 🥳

lajosdani commented 9 months ago

Yes, me too - no changes, just a reinstall and it works OK

I did not even reinstall it. Just left it untouched for weeks, with unavailable error. And now all of a sudden it just started working again.

matilbianco commented 9 months ago

Also here : Integration stopped working once upgraded HA from 10.4 to 11.0. Removed then add HA integration failed with error message : "The connection to the Bosch Indego API failed!" Also tried again after re-installed Chrome extension, same error. From HA Logs :

"Logger: custom_components.indego.config_flow
Source: custom_components/indego/config_flow.py:92
Integration: Bosch Indego Mower ([documentation](https://github.com/jm-73/Indego))
First occurred: 22:27:59 (3 occurrences)
Last logged: 22:31:37"

Error while retrieving mower serial in account! Reason: 403, message='Forbidden', url=URL('https://api.indego-cloud.iot.bosch-si.com/api/v1/alms')

Now working again, just left untouched until yesterday and then reinstalled it.

ArekKubacki commented 9 months ago

Unfortunately, it still doesn't work for me.

Logger: custom_components.indego.config_flow
Source: custom_components/indego/config_flow.py:92
Integration: Bosch Indego Mower ([documentation](https://github.com/jm-73/Indego))
First occurred: 09:32:33 (2 occurrences)
Last logged: 09:33:45

Error while retrieving mower serial in account! Reason: 403, message='Forbidden', url=URL('https://api.indego-cloud.iot.bosch-si.com/api/v1/alms')
sander1988 commented 9 months ago

Sometimes it works, a few days later it's blocked again. I'm pretty sure it's the 'auto-learning' feature of the Azure WAF. :-(

I have added the option to override the user-agent through the HA webinterface. This way the user can override the user-agent header to get around the Azure WAF in case it's blocked again. Without the need of changing the source code every time. It's not perfect, but the best solution I could think of.

Pull requests are open for:

New component setup: image

Existing component > Configure: image

atetzner commented 6 months ago

I am not too deep into the issue, but would it be a "solution" / workaround to use a random user agent?

I assume that the WAF would also block requests, if the user agent changes on every request, but potentially we could perform an automatic retry of requests with a new user agent every time we get a 403.

I also thought about using the same user agent than the official app uses. But yesterday I had the feeling, that the official app also suffers from the 403 problem: I had a large amount of notifications in the app and started to delete them. After deleting 20 notifications or so, the app showed a popup with an error message 15_403 (unsure about the 15 but I definitely saw the 403 and directly remembered this problem here).

atetzner commented 5 months ago

@jlaur : I assume your comment was meant to be posted in the corresponding OpenHab issue? openhab/openhab-addons#16517

However, from the comments in this HomeAssistant issue here, I assume that changing the user agent works to some extend and the possibility for the users to manually adjust the user agent to work around the 403 also seem to work. At least that's what I think as no one posted anything here since start of december ...

jlaur commented 5 months ago

@atetzner - sorry, indeed. I will delete it from there. Thanks for your feedback as well.

ArekKubacki commented 5 months ago

I still have got error. Any idea? Request to 'https://api.indego-cloud.iot.bosch-si.com/api/v1/alms' failed with HTTP status code: 403

leante commented 5 months ago

Same 403 error here. I have tried with several user agent without success.

Core 2024.3.3 Supervisor 2024.03.1 Operating System 11.5 Frontend 20240307.0

Pr0mises commented 5 months ago

Same here again after updating to Core 2024.4.3 Core: 2024.4.3 Frontend 20240404.2

@sander1988 I tried your PR (copied all files from here) but somehow my HASS tries to install 2.2.0 instead of 3.1.3... INFO (SyncWorker_22) [homeassistant.util.package] Attempting install of git+https://github.com/sander1988/pyIndego.git#pyIndego==2.2.0 Yes I triple checked my manifest.json: "requirements": ["git+https://github.com/sander1988/pyIndego.git#pyIndego==3.1.3"],

I already deleted the pycache folder and did a shutdown of my hass docker container and started it again.. No User Agent setting for me... Not sure what's wrong

Nibeck1309 commented 4 months ago

Version: 5.6.1 Core: 2024.4.4 Supervisor: 2024.04.0 Operating System: 12.2 Frontend: 20240404.2

When I tried to add device, I have this issue.


Logger: custom_components.indego.config_flow
Source: custom_components/indego/config_flow.py:92
integration: indego (documentation)
First occurred: April 25, 2024 at 12:27:56 PM (3 occurrences)
Last logged: 9:20:25 AM

Error while retrieving mower serial in account! Reason: 401, message='', url=URL('https://api.indego-cloud.iot.bosch-si.com/api/v1/alms')
sander1988 commented 3 months ago

Some fixes where just merged into #205 regarding the user-agent into the develop branch. That fixed one similar issue that I could reproduce.

I will monitor the stability for the next couple of days. If everything looks fine we can create a new release.

sander1988 commented 3 months ago

I'm closing this issue. We have made several improvements which are currently on the develop branch and being tested. Once there is a new release we can look into connection errors again (in case they still persist).