BMW Connected Drive - 403 Quota Exceeded

[uphillbattle]

The problem

I’m suddenly getting a lot of these:

2022-12-02 12:48:25.497 ERROR (MainThread) [homeassistant.components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-XXX@YYY.ZZ data: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN-redacted/state' 

When this happens, all related entities become unknown (of course). The car can be «lost» for hours, or just minutes, the reappears (for hours or just minutes). Started about 20 h ago.

What version of Home Assistant Core has the issue?

2022.11.4 and 2022.11.5

What was the last working version of Home Assistant Core?

2022.11.4

What type of installation are you running?

Home Assistant OS

Integration causing the issue

BMW Connected Drive

Link to integration documentation on our website

https://www.home-assistant.io/integrations/bmw_connected_drive

Diagnostics information

Not available

Example YAML snippet

No response

Anything in the logs that might be useful for us?

See above

Additional information

No response

[home-assistant[bot]]

Hey there @gerard33, @rikroe, mind taking a look at this issue as it has been labeled with an integration (bmw_connected_drive) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `bmw_connected_drive` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign bmw_connected_drive` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

bmw_connected_drive documentation bmw_connected_drive source _{^{(message by IssueLinks)}}

[gerard33]

I had the same issue today, but it's working fine again. Can you check if the same applies for you? I understood there was a generic issue today with the BMW app as well.

[uphillbattle]

Thanks Gerard, I’m still getting the 403 response. It’s been 9 or 10 hours since last successful contact. With some luck it’ll fix itself, but no luck as of yet. I’ll let you know.

[Mark481971]

I have the same probleme....what can I do?

[uphillbattle]

@gerard33 Regained contact at 1 am (CET). Contact remained stable until 11:10 am, at which time I received another 403 error. After that, I’ve had intermittent contact - but currently not.

[rikroe]

Looks similar to https://github.com/home-assistant/core/issues/78792, just on the next endpoint (the status per vehicle). BMW seems to allow only ~120 requests per day anymore.

If so, the the quota blocking will block your whole account (i.e. also block the official app) and reset at 0:00 UTC.

You can try to disable automated polling (5 minute interval) in the integration'system option settings in HA. Then create an automation that calls the homeassistant.update_entity service every 20 minutes.

[uphillbattle]

@rikroe The API apparently accepts requests intermittently (please see screenshot below), so not quite consistent with blocking until midnight UTC:

Still, I’ll try what you suggest. I have two cars, so how many requests are done per 5 minute interval (one for the list of cars and one for each car, i.e. three, or just one)? In other words, how long between each request is necessary?

[jruibarroso]

I have 2 BMWs, same issue here

[rikroe]

The educated guess is that the cap is 120 requests/day (i.e. 10 hours of requests every 5 minutes). My guess is that this is on a per-endpoint basis, so two cars should be counted seperately (however I'm not sure of this).

So 24 hours/120 requests = 12,5 minutes between the requests. However I would not go down that far, as we sometimes request additional vehicle refreshs (i.e. after service execution).

You might want to create an automation that adjusts the timing manually, e.g. every 30 minutes by default but every 10 minutes while the car is charging (or whatever you're interested in).

Happy to get your feedback what works and what is too much (please try it out!) and let me know so we can adjust the defaults and write some help to the documentation.

[uphillbattle]

@rikroe Thank you for your explanation and input. I will try different timings and report back. Right now I seem to be locked out for the day, so it’s tomorrow and the next days, I guess 😉

[mtthlm]

Just in case it short-circuits someone else's search and debugging, the solution mentioned above and in #78792 seems to have resolved the BMW API rate limit contention issues. Here's what I did:

1. Disable automatic polling for the BMW Connected Drive integration

   Settings -> Devices & Services -> BMW Connected Drive 3-dot menu -> System Options -> Enable polling for updates.

2. Create a new automation, something similar to the following:

alias: BMW Connected Drive Integration Updates
trigger:
  - platform: time_pattern
    minutes: /15
action:
  - service: homeassistant.update_entity
    target:
      entity_id: sensor.330e_xdrive_charging_status
mode: single

Note: Per @RuiSSousa's comment on #78792:

# updating 1 sensor causes all of them to be updated

Note: 15 minutes should be good if you don't care much about manual refreshes, but I've just implemented this workaround today so we'll see how it stands up to the test of time.

:upvote: "HA Enhancement: Integration-specific polling interval configuration option"

[SGXander]

Looks like they have lowered the quota again. Mine started doing this again last night. Last time this happened 5 minutes seemed to be ok now it's more like 10 mins. BMW devs suck

[uphillbattle]

I have disabled the automatic polling and now poll (manually, through an automation) every 30 minutes (I am working on increasing the poll frequency when needed - for example during charging - but haven't finished that yet, so it's every 30 minutes for now). I have not received 403 Quota Exceeded since I implemented the 30 minutes poll period.

However, I'm just wondering: Since setting the (manual) 30 minute poll period, I have noticed that several times a day, I get this error message in the logs:

2022-12-05 12:00:14.948 ERROR (MainThread) [homeassistant.components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-XX@YY.ZZ data: Error communicating with BMW API: Server error '500 Internal Server Error' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN-redacted/state'

How does the integration react to this error when automatic polling is enabled. If automatic polling was enabled, would the integration try again and again until success, and would each try be counted by the API - thereby leading to 403 Quota Exceeded - not because the quota has been reduced but because the retrying leads to much more API requests? Could the server side problem (leading to the 500 response) be the root cause of the 403 responses we've seen lately instead of an actual quota reduction from BMW?

[RuiSSousa]

Since I'm not having issues, I'm going to share my config:

I have three automations and automatic polling disabled. I detect if the car is charging using a power meter, but you can use the car charging status (enable the disabled conditions and disable the enabled ones):

1 - Refresh while charging:

alias: 330e Refresh From Cloud While Charging
description: ""
trigger:
  - platform: time_pattern
    minutes: /5
condition:
  - condition: state
    entity_id: sensor.330e_charging_status
    state: CHARGING
    enabled: false
  - condition: numeric_state
    entity_id: sensor.330e_charger_power
    above: 1000
action:
  - service: homeassistant.update_entity
    data: {}
    target:
      entity_id: sensor.330e_charging_level_hv
mode: single

2 - Refresh while not charging:

alias: 330e Refresh From Cloud While Not Charging
description: ""
trigger:
  - platform: time_pattern
    minutes: /20
condition:
  - condition: state
    entity_id: binary_sensor.330e_charging_status
    state: "off"
    enabled: false
  - condition: numeric_state
    entity_id: sensor.330e_charger_power
    below: 900
action:
  - service: homeassistant.update_entity
    data: {}
    target:
      entity_id: sensor.330e_charging_level_hv
mode: single

3 - Reload when unavailable for over 15 minutes because a sensor update sometimes wouldn't work after service outage:

alias: 330e Reload Integration.
description: ""
trigger:
  - platform: state
    entity_id:
      - sensor.330e_charging_end_time
    to: unavailable
    for:
      hours: 0
      minutes: 15
      seconds: 0
condition: []
action:
  - service: homeassistant.reload_config_entry
    data: {}
    target:
      device_id: 801b792cf2ed4af31d2bee9c2d731957
mode: single

Hope it helps

[yozh]

Very interesting that this started happening to me about 3 days ago, while I go the car on Nov 21st and setup a lot of automations including refreshing every 10 seconds as Im trying to catch when doors are closed/open since we cant identify when engine starts, also see if windows are staying open etc, but even with 10 sec refreshed it worked half the time to catch anything. but it was polling fine, until 3 days ago when I started to get this same error. Also seems that App works, but Im almost certain not from my home IP… I really dont like how this integration is playing out and it’s totally due to the way BMW is implementing this … volvo integration was sooooo much better…. I guess Ill be playing around with this now. No one knows the magic number ?

[uphillbattle]

Annoying as it may be, that we cannot request updates from the API as often as we like, let’s look at it from BMW’s perspective:

If I’m not mistaken, we’re exploiting an API that BMW has set up for the benefits of their customers using the MyBMW app (correct me if I’m wrong). The infrastructure required to provide this service requires investments by BMW.

BMW has decided to provide another API for which they charge the users (https://bmw-cardata.bmwgroup.com/thirdparty/public/car-data/technical-configuration/api-documentation). If everyone instead of paying for the use of that commercial API, exploits the MyBMW app API, they lose income and have to invest more in the MyBMW app API.

Looking at it this way, I can’t really blame BMW for limiting the number of API calls per day. Even the commercial API has a limit, although that limit is 100 requests per minute.

I am now updating my BMW entities like this:

• Every half hour generally
• Every 10 minutes when charging
• I make single calls whenever I plug the car into the charger (my charger has an open API that the supplier encourages people to use, so I get instant (push) notification from the charger when the car is plugged in).
• I make single calls every 10 minutes if BMW entities are unavailable

I am counting the calls, and so far today (since midnight UTC) I have 63 calls. I will probably end up at about 75 or so before midnight UTC. I could probably limit it even more, without losing the functions I’m after (I mainly use the BMW data to control smart charging during less expensive electricity hours and to start climatizing the car before departure).

So, if the quota stays at 120 per day (which has been suggested), I should be fine. 😄

[yozh]

So this is where this gets interesting. Im home now. The integration doesn't work. But app is updating and showing current info from same network.....

HA for me shows this..... "Retrying setup: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.us/eadrax-vcs/v2/vehicles' "

[rikroe]

HA for me shows this..... "Retrying setup: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.us/eadrax-vcs/v2/vehicles' "

This happens if your HA is trying to restart/create the integration too many times. Will also reset at 0:00 UTC and was 120 requests a day before (but maybe that changed as well).

[robertwigley]

If I’m not mistaken, we’re exploiting an API that BMW has set up for the benefits of their customers using the MyBMW app (correct me if I’m wrong). The infrastructure required to provide this service requires investments by BMW.

Some customers like me are paying about £120 a year for ConnectedDrive Professional services. This is where it becomes irksome.

[yozh]

HA for me shows this..... "Retrying setup: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.us/eadrax-vcs/v2/vehicles' "

This happens if your HA is trying to restart/create the integration too many times. Will also reset at 0:00 UTC and was 120 requests a day before (but maybe that changed as well).

Why only from HA and not the app?

[uphillbattle]

If I’m not mistaken, we’re exploiting an API that BMW has set up for the benefits of their customers using the MyBMW app (correct me if I’m wrong). The infrastructure required to provide this service requires investments by BMW.

Some customers like me are paying about £120 a year for ConnectedDrive Professional services. This is where it becomes irksome.

Good point 😉

[SGXander]

HA for me shows this..... "Retrying setup: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.us/eadrax-vcs/v2/vehicles' "

This happens if your HA is trying to restart/create the integration too many times. Will also reset at 0:00 UTC and was 120 requests a day before (but maybe that changed as well).

Why only from HA and not the app?

Same for me, when HA is getting 403s the app is ok. It likely means BMW are using something more complex than just IP address or account to quota the api calls. If they provide a client id when authenticating it may be possible to spoof a "new" login to get around this?

[yozh]

HA for me shows this..... "Retrying setup: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.us/eadrax-vcs/v2/vehicles' "

This happens if your HA is trying to restart/create the integration too many times. Will also reset at 0:00 UTC and was 120 requests a day before (but maybe that changed as well).

Why only from HA and not the app?

Same for me, when HA is getting 403s the app is ok. It likely means BMW are using something more complex than just IP address or account to quota the api calls. If they provide a client id when authenticating it may be possible to spoof a "new" login to get around this?

Thats what Im thinking, there must be some kind of cashed token presented which is how it knows. But this is way above my level :)

[rikroe]

Ok, that is an interesting observation!

One thing that might change (or how the API might differentiate) is the bmw-session-id header which seems to be recreated everytime the app is started.

The library kind of does this as well, if it is completely instantiated from scratch.

Has any of you tried just disabling/reenabling the integration after you hit the quota limit? This should recreate the session id.

[yozh]

Ok, that is an interesting observation!

One thing that might change (or how the API might differentiate) is the bmw-session-id header which seems to be recreated everytime the app is started.

The library kind of does this as well, if it is completely instantiated from scratch.

Has any of you tried just disabling/reenabling the integration after you hit the quota limit? This should recreate the session id.

Yes, disabling reenabling or even restarting HA doesnt clear it up.

[rikroe]

Then it seems this is not the identifier used to differentiate between devices or it is in combination with something else.

[yozh]

What can we do to try to get what it is using ?

Im surprised more people are not complaining, because with normal pre set values of polling it does not last for a day, about 3 or so hours before 0 UTC it starts failing for me. What Im really curios about is why this worked for about 2 weeks with out hitting the quota... There must be something else involved.

[SGXander]

Working for a time could just be explained away by BMW changing the limit or the detection mechanism to try and stop us using it.

The other possibility is BMW are doing it manually or semi-automatically i.e. they run a script or schedule a script to go through and disconnect any connections that have a high count at certain times. if that were true then it would still need a spoof of a "new" client each time though...

To find out if it is instance and/or IP related it would be something like add the bmw connected drive integration on a separate version of HA like a casa instance. disable it and then enable it when the first one stops working. Assuming that works then it is at least bypassable. Gathering full logs when doing that will probably help rikroe figure it out. It can only be based on a client ID given to it on auth so the question is how is this made. it must be based on IP and probably account and time but it wouldn't be wide IP or your phone app wouldnt work when HA stops so something else. I'll get wireshark going on HA and my phone next week to see if there's anything obviously different about them.

Such a shame that so many companies are doing this with their APIs now. More annoying still that we pay for connected drive services. Not like you could ever speak to someone at BMW to ask them to stop either...

Long post almost over but one last thought: BMW have changed (at least in the UK) to My BMW from the connected drive app. could just be a rebrand but it is possible they are toning down the connected drive API becuase it is being phased out?

[yozh]

Everything you listed I have been pondering about. Trying to see where else I can run HA on another IP when this stops working, I have captured HA traffic to the BMW servers, but it is HTTPs and I cant really tell what API is doing or how the URL/Call is formed looks like. But thats really just a quick glance.. The app goes to the same place as HA and flow looks basically same.

I was also thinking they changed it for all users, but its possible they do analytics on high calls requests, again there should be something identifying this, Im just a bit puzzled that out of 1336 (reported) integrations only few people are complaining, with default settings not lasting a whole day.

I also noticed something else, which Im not sure if I saw App doing before, but I only have this car for few... But when there is no activity in the car, the apps last status is something like this "Updated from vehicle 12/10/2022 2:07 PM" This was refreshed in app few times at 5:41 PM, so maybe something changed on the way App uses the API and doesn't pull if there are no changes.

[SGXander]

The app goes to the same place as HA and flow looks basically same.

So it's unlikely the api address is changed/deprecated that's one thing off the list of possibles.

Im just a bit puzzled that out of 1336 (reported) integrations only few people are complaining, with default settings not lasting a whole day.

I don't know how they judge this number but I do know a friend who has the integration but doesn't really use it yet so may not have noticed.

I also noticed something else, which Im not sure if I saw App doing before, but I only have this car for few... But when there is no activity in the car, the apps last status is something like this "Updated from vehicle 12/10/2022 2:07 PM" This was refreshed in app few times at 5:41 PM, so maybe something changed on the way App uses the API and doesn't pull if there are no changes.

This is how it has always worked as far as I can remember. The server maintains a state of last pushed data so it doesn't really have a way of calling the car up, rather the car periodically sends information but mostly only sends it when something changes. It's likely the reason BMW are giving us a hard time about it because repeated "refreshes" from an "app" would look strange given there haven't been any updates from the car. what I'm not sure about is how you can send commands from the app, via server to the car. I think it periodically dials home to see if there are any tasks or perhaps registers for a type of push notification of its own. None of this helps us but if there is a "last received update" api call that sits outside of the quota limit then rikroe could use that to detect changes and only update the sensors/entities if the timestamp has changed...

[robertwigley]

Thinking outside the box, I am considering using the state of a template sensor for the bluetooth connection of my phone to the car to determine when to request polling i.e. the car is in use with someone in it. Maybe this will help until a more permanent solution is found.

[arboeh]

I don't know if it helps: I'm using the EVCC addon for HA (https://github.com/evcc-io/evcc) with my BMW also and it still works with the BMW API. I had connection problems and "403"' in the past also and andig (https://github.com/andig) solved them. Maybe he can help to solve this because of his experiences.

[SGXander]

I don't know if it helps: I'm using the EVCC addon for HA (https://github.com/evcc-io/evcc) with my BMW also and it still works with the BMW API. I had connection problems and "403"' in the past also and andig (https://github.com/andig) solved them. Maybe he can help to solve this because of his experiences.

Do you have the issue number from when you had 403s? It might show how andig solved it.

[yozh]

I think this is it….

https://github.com/evcc-io/evcc/issues/3525

Seems like a user agent change might help resolve this?

[arboeh]

There have been more than one issue with the sh... BMW API: https://github.com/evcc-io/evcc/issues?q=BMW+403+ Hope it helps.

[yozh]

I also saw this: “ With respect to the cocoapi: yes, it was not meant to be publicly available but accessed by BMW applications via public networks (in contrast to vpn like vehicle to backend telematics channels). The intention from security perspective is to block old MyBMW app releases which might and will become vulnerable over time.”

it’s possible since app update the user agent changed.

I wonder how we can update the user agent our selfs

[yozh]

So looking thru the code, I see that currently its using Android Version of 2.5.2 and we are already on 2.11.1 is it possible they are limitting the old version requests ?

APP_VERSIONS = { Regions.NORTH_AMERICA: "2.5.2(14945)", Regions.REST_OF_WORLD: "2.5.2(14945)", Regions.CHINA: "2.3.0(13603)",

[yozh]

Anything on this ? No one seems to care too much or only few people are getting this same errors, which I find strange..

[RuiSSousa]

Anything on this ? No one seems to care too much or only few people are getting this same errors, which I find strange..

Only getting the occasional 500 here...

[robertwigley]

I suspect everyone is getting the error and we're all just waiting for it to be fixed by someone who understands it. The integration has broken that many times recently (due to changes on BMWs side), that people are probably not complaining about it so much and just expect it will start working again once someone figures out how to fix it.

[uphillbattle]

I am now updating my BMW entities like this:

• Every half hour generally
• Every 10 minutes when charging
• I make single calls whenever I plug the car into the charger (my charger has an open API that the supplier encourages people to use, so I get instant (push) notification from the charger when the car is plugged in).
• I make single calls every 10 minutes if BMW entities are unavailable

As mentioned above, I am now polling as described in the quote. I'm polling about 70-80 times per 24 hours. Since I started polling like this (last week), I have not seen the 403 error, just the occassional 500 like @RuiSSousa mentions.

[yozh]

I am now updating my BMW entities like this:

• Every half hour generally
• Every 10 minutes when charging
• I make single calls whenever I plug the car into the charger (my charger has an open API that the supplier encourages people to use, so I get instant (push) notification from the charger when the car is plugged in).
• I make single calls every 10 minutes if BMW entities are unavailable

As mentioned above, I am now polling as described in the quote. I'm polling about 70-80 times per 24 hours. Since I started polling like this (last week), I have not seen the 403 error, just the occassional 500 like @RuiSSousa mentions.

Probably works for your needs, I need bit more closer to real time. Currently trying to do /7 minutes to see how that goes. Tried manually changing the version (not sure if I did it right) but I dont think it helps, we are now way behind at 2.5 in integration and 2.12 on app stores.