BMW Connected Drive error 429

[silviudc]

The problem

BMW connected drive integration is getting an error 429 for about 1 day or so. And the log keeps showing this error.

What version of Home Assistant Core has the issue?

core-2022.6.5

What was the last working version of Home Assistant Core?

core-2022.6.4

What type of installation are you running?

Home Assistant OS

Integration causing the issue

BMW Connected Drive

Link to integration documentation on our website

https://www.home-assistant.io/integrations/bmw_connected_drive/

Diagnostics information

No response

Example YAML snippet

N/A

Anything in the logs that might be useful for us?

2022-06-11 19:49:20 WARNING (MainThread) [homeassistant.config_entries] Config entry 'xxx@xxx.com' for bmw_connected_drive integration not ready yet: Error communicating with BMW API: Client error '429 Too Many Requests' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=60.0&appDateTime=1654973346008&tireGuardMode=ENABLED'

Additional information

Nope

[probot-home-assistant[bot]]

bmw_connected_drive documentation bmw_connected_drive source _{^{(message by IssueLinks)}}

[probot-home-assistant[bot]]

Hey there @gerard33, @rikroe, mind taking a look at this issue as it has been labeled with an integration (bmw_connected_drive) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[rikroe]

It seems our timeouts are not long enough. As a workaround, please the following: deactivate the integration, wait an hour and activate it again.

Please see the workaround below in https://github.com/home-assistant/core/issues/73370#issuecomment-1159060882.

[silviudc]

It seems our timeouts are not long enough. As a workaround, please the following: deactivate the integration, wait an hour and activate it again.

Will try this right now. Thanks

[silviudc]

No luck, same error 429 after that

[RuiSSousa]

Same here. I frequently have error 429 but reconnects after a while. Is there any way to lower the request rate? Thanks.

[z-vrba]

Same error permanently.

[jmvaz]

Same :(

[olesj33]

Same here .

[silviudc]

Leave it disabled for longer.

[RuiSSousa]

I may have found a work around:

Disable "Enable polling for updates." in the BMW integration System Options

Create an automation to run "Press 330e Refresh from cloud button" every 5 minutes or so.

Done!

Not having issues since last night.

Edit: Scratch that. 429 again.

[BinnenBeest]

I may have found a work around:

Disable "Enable polling for updates." in the BMW integration System Options

Create an automation to run "Press 330e Refresh from cloud button" every 5 minutes or so.

Done!

Not having issues since last night.

Edit: Scratch that. 429 again.

Tried it the same way, worked till I asked for to refresh. So I have the same issue.

[rikroe]

Ok, somehow the integrations seems to spam the BMW APIs. With a test setup in rest_of_world, I can call the API 8 times and only get the 429 error anywhere on the 4th to 8th try. On the 8th try, the API responds with { "statusCode": 429, "message": "Rate limit is exceeded. Try again in 224 seconds." }, i.e. about 4 minutes, which should perfectly work with the 5 minute refresh interval in HA.

Given this, the workaround should be as @RuiSSousa mentioned in https://github.com/home-assistant/core/issues/73370#issuecomment-1157417246: Disable automatic polling, wait at least 5 minutes (or maybe up to a day, don't know if it stacks up. Then re-enable polling again.

Completely deactivating the integration was the wrong way to do, as this also calls the BMW API twice, possibly adding to the issue.

[RuiSSousa]

It seems to work indefinitely until using the MyBMW app, the integration then throws a 429. Reloading the integration after a few minutes reconnects. Doesn't seem to stack up. Increasing the refresh to 15 minutes had no effect.

Will this issue be solvable from our end?

[andreas-p]

I wonder if the issue is somehow tied to the User-Agent.

[RuiSSousa]

OK, full workaround:

1 - Disable "Enable polling for updates." in the BMW integration System Options

2 - Create an automation to run "Press Refresh from cloud button" every 5 minutes or so. 10 or 15 minutes seems to work better.

3 - Create an automation to reload the integration config entry after 5 minutes of any sensor becoming "Unavailable".

Caveats:

It will still become unavailable from time to time, whenever the MyBMW app is used it's an almost certain 429 in the integration, but the reloading automation will take care of that.

Edit: Maybe just reloading the config every 5 minutes is also a viable/better workaround? Can @rikroe shed some light? I know nothing of HA or integrations inner workings.

[Mithrox]

Just so I understand this correctly, step 2 would be setting up an automation like this below?

'alias: BMW reload description: '' trigger:

• platform: time_pattern minutes: '10' condition: [] action:
• service: homeassistant.reload_config_entry data: entry_id: id_from_core.config_entries mode: single'

[rikroe]

Reloading the config every 5 minutes is exactly the wrong thing to do, this causes too many requests to to API. If you want to automate it, you can just call homeassistant.update_entity on any sensor/button/..., this will trigger just the data refresh.

I also have just finished a PR that should improve the situation a lot - if the update fails it will only log a warning (with the time until a retry is possible) and without raising an error, which should keep the existing values/buttons/... intact.

[RuiSSousa]

@Mithrox

platform: time_pattern minutes: /10

It needs to be /10 to run every 10 minutes.

Sorry for the garbled comment

[rikroe]

@RuiSSousa you might want to clean your last comment a little bit, seems like Outlook screwed this up nicely.

[Mithrox]

Reloading the config every 5 minutes is exactly the wrong thing to do, this causes too many requests to to API. If you want to automate it, you can just call homeassistant.update_entity on any sensor/button/..., this will trigger just the data refresh.

I also have just finished a PR that should improve the situation a lot - if the update fails it will only log a warning (with the time until a retry is possible) and without raising an error, which should keep the existing values/buttons/... intact.

Ok, so if I then replace the reload with the update_entity like this, I just chose the charging_level.

action:

• service: homeassistant.update_entity data: {} target: entity_id: sensor.x5_xdrive40e_iperformance_charging_level_hv

As a follow-up question, will updating one sensor update the rest by default, or do I need to run updates on sensors regularly on all of them? Ideally you want all to update at least sometimes regularly.

[RuiSSousa]

@rikroe Thanks a lot.

[Mithrox]

@Mithrox platform: time_pattern minutes: /10 It needs to be /10 to run every 10 minutes. Sorry for the garbled comment

Ah okay. I used it previously but I saw there was now a time_pattern option and I thought just putting 10 in would work fine. But I will change it, thank you.

[rikroe]

Updating one sensor will update all sensors of the integration.

[Mithrox]

I am running the 10 minute update entity like shown above, but the only thing that updates on the car, is the Location (top item). All the others have still not updated the last hour, even though the automation has ran every 10 minutes, and another time now manually to check again. Any idea why this may be the case?

[rikroe]

@Mithrox can't tell you directly why this is happening. For now, use the button service:

service: button.press
target:
  entity_id: button.VEHICLE_refresh_from_cloud

[Mithrox]

So like this?

service: button.press
target:
  entity_id: button.x5_xdrive40e_iperformance_refresh_from_cloud

I don't know why I can never use code formatting, it never comes up the way you did above here :)

Edit: Got it, thank you for the tip below, that makes it so much easier.

[rikroe]

Yes. You can also use the visual editor in Home Assistant to just select it.

To format code, you have to use markdown syntax: https://www.markdownguide.org/extended-syntax/#fenced-code-blocks

[rikroe]

It seems to work indefinitely until using the MyBMW app, the integration then throws a 429. Reloading the integration after a few minutes reconnects. Doesn't seem to stack up. Increasing the refresh to 15 minutes had no effect.

Will this issue be solvable from our end?

Just found out yesterday that the BMW App is using different API endpoints (there's a V2 for MyBMW apparently). Probably they are limiting the old endpoints for this reason that much.

[Mithrox]

@Mithrox can't tell you directly why this is happening. For now, use the button service:

service: button.press
target:
  entity_id: button.VEHICLE_refresh_from_cloud

It is weird, even with this one I only get update on the Location, not on any of the others. However the car has not moved, been charged or anything, so it may not "update" until the actual change happens. I am trying with both "Last Updated" and "Last Changed", but it still remains no change to the "timestamp" with either of these updates.

[Mithrox]

Just found out yesterday that the BMW App is using different API endpoints (there's a V2 for MyBMW apparently). Probably they are limiting the old endpoints for this reason that much.

You think the integration here will be able to use the new API over time, or is BMW doing this to limit the traffic or use of their APIs?

[rikroe]

It is weird, even with this one I only get update on the Location, not on any of the others. However the car has not moved, been charged or anything, so it may not "update" until the actual change happens. I am trying with both "Last Updated" and "Last Changed", but it still remains no change to the "timestamp" with either of these updates.

Please keep an eye on this and (if possible) test with both after the vehicle has moved a bit. Don't know how accurate it is.

You think the integration here will be able to use the new API over time

We should be.

or is BMW doing this to limit the traffic or use of their APIs?

Both.

[Mithrox]

It is weird, even with this one I only get update on the Location, not on any of the others. However the car has not moved, been charged or anything, so it may not "update" until the actual change happens. I am trying with both "Last Updated" and "Last Changed", but it still remains no change to the "timestamp" with either of these updates.

Please keep an eye on this and (if possible) test with both after the vehicle has moved a bit. Don't know how accurate it is.

I've now setup 2 cards, where one shows Last Changed on each entry, and the other showing Last Updated. I will see if there is any difference. However, when I used the refresh_from_cloud button every 10 minutes, the integration went into error quickly, I had to reload it and then change back to the previous method of refreshing a sensor.

I will try both more, but not until I drive the car, I'll change it while I am out in the car.

[sjauquet]

Hello, Even with 15 minutes polling, the problem appears after some hours. Once it's "unavailable" I have to reload the integration to have it available again. As from the api documentation, we should have a quota of 100 requests per minute... https://bmw-cardata.bmwgroup.com/thirdparty/public/car-data/technical-configuration/api-documentation Is the module flooding the BMW API ? (I have disabled "Enable polling for updates." in the BMW integration System Options) Maybe it happens when I use the car, a that moment, the car send datas to the api and one call from HASS is too much and locks the module.

This is very annoying. I have an I3 Please fix it, Thank you very much !

[andreas-p]

I guess in case of 429 the auth_token has to be discarded, and a fresh token retrieved via login().

[rikroe]

I guess in case of 429 the auth_token has to be discarded, and a fresh token retrieved via login().

During my testing, it seemed as if the limit was placed on the account and a new login didn't solve the issue. Only waiting did...

Is the module flooding the BMW API ?

No, we're doing two requests every five minutes. But seems like we're using an older API and BMW generally doesn't seem to like us using their API. Also, we're not using the CarData API provided by BMW as you have to pay for it but the APIs that power the MyBMW app directly (by reverse engineering, so no documentation etc. and it's a lot of poking in the dark).

[andreas-p]

@rikroe We probably need a combination of waiting and fresh login. It appears as if the api will remain in the 429 state for the old session, so a sleep with the old token won't heal the problem.

[Mithrox]

@rikroe We probably need a combination of waiting and fresh login. It appears as if the api will remain in the 429 state for the old session, so a sleep with the old token won't heal the problem.

I've gotten the 429 since early yesterday and after testing a bit I disabled the whole integration last night, left it off all night, but when I tried to start it again this morning I got the 429 right away. Is there some "reset" that can be done to the account/API access, or is it just generally stuck for many of us now? This is the thing you're talking about here right?

[andreas-p]

Hm there's something special about the Mini app. I drove the API into 429 using bimmerconnected, and then started the Mini app. From the display, it seemed to perform some kind of retry, and then showed the status happily. bimmerconnected still got 429 immediately after.

[sjauquet]

(I'm not sure but things have improved since yesterday, no more "unavailable" and I didn't change anything ? Maybe a "server" misconfiguration BMW side ? Anyone can confirm this ? (I've lowered the refresh to 10 minutes since yesterday midnight)

[andreas-p]

No improvement here.

[rikroe]

To give you an update: As https://github.com/home-assistant/core/pull/73675 was merged and is targeted for 2022.6.8, the amount of calls is further reduced when setting the integration up the first time. Also, the integration now automatically adjusts the next polling interval to 15 minutes if a 429 error occurred.

On the long run, https://github.com/bimmerconnected/bimmer_connected/pull/462 in the library will provide a better solution as we switch to the newer API, which hopefully doesn't have the same limitation as the current one (which seems to be deprecated).

On a sidenote here - if you have a convertible, please help us by posting a fingerprint in https://github.com/bimmerconnected/bimmer_connected/pull/462. Thats the last open point.

[z-vrba]

New error in the log. Not sure if this is somehow connected with our topic.

`Logger: homeassistant.components.bmw_connected_drive.coordinator Source: components/bmw_connected_drive/coordinator.py:55 Integration: BMW Connected Drive (documentation, issues) First occurred: 20. června 2022 19:10:43 (58 occurrences) Last logged: 22. června 2022 21:04:26

Unexpected error fetching bmw_connected_drive-XX@XX data: 'reductionOfChargeCurrent' Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 191, in _async_refresh self.data = await self._async_update_data() File "/usr/src/homeassistant/homeassistant/components/bmw_connected_drive/coordinator.py", line 55, in _async_update_data await self.account.get_vehicles() File "/usr/local/lib/python3.9/site-packages/bimmer_connected/account.py", line 84, in get_vehicles existing_vehicle.update_state(vehicle_dict) File "/usr/local/lib/python3.9/site-packages/bimmer_connected/vehicle/vehicle.py", line 109, in update_state curr_attr.update_from_vehicle_data(vehicle_data) File "/usr/local/lib/python3.9/site-packages/bimmer_connected/models.py", line 36, in update_from_vehicle_data parsed = self._parse_vehicle_data(vehicle_data) or {} File "/usr/local/lib/python3.9/site-packages/bimmer_connected/vehicle/charging_profile.py", line 119, in _parse_vehicle_data retval["preferred_charging_window"] = ChargingWindow(charging_profile["reductionOfChargeCurrent"]) KeyError: 'reductionOfChargeCurrent' `

[rikroe]

The KeyError is a separate issue that will be fixed with #73888.