bmw_connected_drive - 403 Quota Exceeded

[Cebeerre]

The problem

Seems that BMW has limited the amount of queries you can do to their API, as it's now throwing a 403 with a "Quote Exceeded" message making all the sensors unavailable.

What version of Home Assistant Core has the issue?

2022.9.5

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

bmw_connected_drive

Link to integration documentation on our website

https://www.home-assistant.io/integrations/bmw_connected_drive/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2022-09-19 19:43:30.442 ERROR (MainThread) [homeassistant.components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-XXXX@XXXX.com data: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles'
For more information check: https://httpstatuses.com/403
2022-09-19 19:53:36.015 ERROR (MainThread) [homeassistant.components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-XXXX@XXXX.com data: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles'
For more information check: https://httpstatuses.com/403
2022-09-19 20:13:40.208 ERROR (MainThread) [homeassistant.components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-XXXX@XXXX.com data: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles'
For more information check: https://httpstatuses.com/403
2022-09-19 20:28:44.379 ERROR (MainThread) [homeassistant.components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-XXXX@XXXX.com data: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles'
For more information check: https://httpstatuses.com/403
2022-09-19 20:58:47.330 ERROR (MainThread) [homeassistant.components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-XXXX@XXXX.com data: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles'
For more information check: https://httpstatuses.com/403
2022-09-19 21:13:50.213 ERROR (MainThread) [homeassistant.components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-XXXX@XXXX.com data: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles'

Additional information

No response

[BeerDiet]

Typically, the HA user is the owner of the car, thus already paying for BMW Connected Drive, trying to retrieve his own data...

For this specific API, they only offer that through their sanctioned apps. I can see their point. My company offers tons of APIs commercially and they were built to a certain spec in that we expect humans to be consuming them and we know the expected usage metrics around them. We also have public-facing APIs that are backends to front-end applications. I'd be none too pleased if I found a customer leveraging those backend APIs for the intended purpose of the APIs we are selling. I'm sure it happens, though.

[rikroe]

To give you a quick update: the chagnes from 2022.9.6 enable me to poll every five minutes, starting yesterday morning 9:00 UTC+2 until now without any issues.

We'll see how long that will work :)

[Zixim]

People using the Viessmann Vicare HA integration have been running into the same problems. Viessmann are rate limiting the API :

The Viessmann API is rate-limited. If you exceed one of the limits below you will be banned for 24 hours:

Limit 1: 120 calls for a time window of 10 minutes
Limit 2: 1450 calls for a time window of 24 hours

Exceeding the limits will (temporarily) break the integration as well as the official Viessmann mobile app. The most elegant solution is to disable the polling in the integration settings, and then make an automation that handles the polling. On a schedule that is convenient for you.

Using this same idea on the BMW Connected Drive has been working just fine for me.

I'm using 2 automations to handle the polling (as I am with my Vicare setup) :

• one for day-time updates, from 07:00:00 to 23:59:00 which refreshes every 15 minutes.
• one for night-time, from 00:00:00 to 07:00:00 which refreshes every hour.

The reasoning for 2 different frequencies is pretty obvious, when I'm asleep idgaf about having fresh data. And it creates a bit of a buffer during the daytime, for app usage.

This comment is getting a little long, I'll just drop my automation for daytime refreshing :

automation:
  - id: f55447ea79875126
    alias: update_ix3_stats_day
    description: Update BMW ix3 entities every 15 minutes
    initial_state: on
    trigger:
      - platform: time_pattern
        minutes: /15
    condition:
      alias: "Time 0700~2359"
      condition: time
      after: "07:00:00"
      before: "23:59:00"
    action:
      - service: homeassistant.update_entity
        data:
          entity_id: sensor.ix3_m_sport_charging_status
          # updating 1 sensor causes all of them to be updated
    mode: single

PS: Not documenting the exact limits is a bit of a dick move by BMW, but I can understand...since they aren't endorsing public API usage (which is also a dick move 😛 ).

[TheGabeMan]

Yesterday 18:00hrs updated to Home Assistant 2022.9.6. Today 10:37am still no issues.

[SGXander]

same here, 2022.9.6 seems to resolve. Sorry if it's mentioned above but should we be wary of update frequency? I have automations to auto-lock the cars etc that may need tweaking if the updates are manual/triggered now.

[Zixim]

should we be wary of update frequency

This entire issue is about polling frequency vs API rate limiting , resulting in the integration being banned until the counter resets. If your integration gets banned, HA has no way of knowing anything about the car. How does your auto-lock automation handle this case ?

[SGXander]

at the moment it doesn't, that's where I was coming from in asking. I'll add an unknown check to it but if the update with the unlocked status doesn't appear then it will likely never run. I'm thinking a warning notification to let me know if the cars go unknown so at least I can be aware...

[Zixim]

@SGXander I'm thinking you'd need to check for status unavailable, and react to that. You won't be able to auto-lock, bc the actions will also be unavailable 😢

[SGXander]

Ah of course the whole api gets locked out. What a pita move from BMW... Adding a notification for unavailable then which will at least tip me off. Thanks @Zixim last stupid question: what is the current interval 5 mins?

[Zixim]

current interval 5 mins?

I think so... a dev would know so. My interval is 20 minutes 😄

[SGXander]

So do we have to control it via an update automation otherwise it's manual?

[Zixim]

Yes, there is no parameter to the integration for setting a custom interval. This is a general HA limitation. Meaning that, to set a custom interval, you must stop HA from auto-polling, and poll on your own schedule through an automation. (Which is a really flexible way to handle this)

[RuiSSousa]

People using the Viessmann Vicare HA integration have been running into the same problems. Viessmann are rate limiting the API :

The Viessmann API is rate-limited. If you exceed one of the limits below you will be banned for 24 hours:

Limit 1: 120 calls for a time window of 10 minutes
Limit 2: 1450 calls for a time window of 24 hours

Exceeding the limits will (temporarily) break the integration as well as the official Viessmann mobile app. The most elegant solution is to disable the polling in the integration settings, and then make an automation that handles the polling. On a schedule that is convenient for you.

Using this same idea on the BMW Connected Drive has been working just fine for me.

I'm using 2 automations to handle the polling (as I am with my Vicare setup) :

• one for day-time updates, from 07:00:00 to 23:59:00 which refreshes every 15 minutes.
• one for night-time, from 00:00:00 to 07:00:00 which refreshes every hour.

The reasoning for 2 different frequencies is pretty obvious, when I'm asleep idgaf about having fresh data. And it creates a bit of a buffer during the daytime, for app usage.

This comment is getting a little long, I'll just drop my automation for daytime refreshing :

automation:
  - id: f55447ea79875126
    alias: update_ix3_stats_day
    description: Update BMW ix3 entities every 15 minutes
    initial_state: on
    trigger:
      - platform: time_pattern
        minutes: /15
    condition:
      alias: "Time 0700~2359"
      condition: time
      after: "07:00:00"
      before: "23:59:00"
    action:
      - service: homeassistant.update_entity
        data:
          entity_id: sensor.ix3_m_sport_charging_status
          # updating 1 sensor causes all of them to be updated
    mode: single

PS: Not documenting the exact limits is a bit of a dick move by BMW, but I can understand...since they aren't endorsing public API usage (which is also a dick move 😛 ).

I'm doing something similar:

Update every hour under normal conditions and every 15 minutes while charging, because I like to keep the batteries under 80 maximum SoC and BMW removed the ability in iDrive for my PHEV. I detect if the vehicle is charging and turn on/off the power with a smart socket.

[SGXander]

One for the dev then maybe but is it still true that any entity refresh will pull the whole payload for the connecteddrive account? I just want to know if I need to refresh one entity per car or just any from BMW to update the lot every few minutes.

[Zixim]

any entity refresh will pull the whole payload for the connecteddrive account

This is the way. I'm only refreshing sensor.ix3_m_sport_charging_status, yet all entities populate.

[SGXander]

gotcha I'm trying /5 mins with no conditions which hopefully works out well enough. The cars are in use throughout the day as my Wife works nights too so the dont care period for me is only about 2 or 3 hours in the morning...

[Zixim]

@SGXander don't forget to disable polling in the integration's System options.

[sfortis]

Yes, there is no parameter to the integration for setting a custom interval. This is a general HA limitation. Meaning that, to set a custom interval, you must stop HA from auto-polling, and poll on your own schedule through an automation. (Which is a really flexible way to handle this)

A universal mechanism in HA allowing to specify the polling rate (in integration system options for example) will resolve many issues and this should be a great enhancement IMO, since till now is pretty much hardcoded.

[rhoddan]

One variant can of course be to add an update button (in combination to deactivate the polling). Not perfect but it will limit the need for API requests. You can of course have lets say 1 hour polling in combination.

[olivierdj]

While technically true (as far as I understand the API & quotas), I don't think it using two phones will have that big of an impact. The MyBMW apps only poll the APIs when opened, so you do not create the same amount of calls as we do (or did before 2022.9.6).

I don't think it works that way for the app. With the app on the background I get notifications as soon as charging starts and ends for example. I would say that the app polls the status on a regular basis. Probably not as much as the HA integration, but still...

[Zixim]

BMW could also be pushing this type of info to the app.

[rhoddan]

Is it possible to get a trigger when someone open a specific dashboard containing bmw related sensors (browser or app). If so you can request for an update?

[rikroe]

The integration currently uses a 5 minute polling interval (see bmw_connected_drive/coordinator.py#L19).

If that doesn't work for you, it can always be changed. However the last change seemed help, as apparently only the "list vehicles" endpoint has such a strict quota - that may change in the future of course.

Otherwise @Zixim's automation (https://github.com/home-assistant/core/issues/78792#issuecomment-1255926045) can always be used to fine-tune the polling interval (disabling the global polling of course). Also you can do much more, depending on car location, charging status etc.

I confirm that to refresh ALL sensors, you just need to update one.

[rhoddan]

Maybe it mentioned somewhere - but is the automatic poll setting on/off under configure gone?

[Zixim]

@rhoddan automatic polling is located on the integration's card > Kebab menu > System options

[rhoddan]

@rhoddan automatic polling is located on the integration's card > Kebab menu > System options

Ahh. My bad - I thought it was under configure

[rikroe]

After using 2022.9.6 it for almost 48 hours, the 5 minute polling interval continues to work for me.

However I am thinking about adding a paragraph for custom polling intervals to the docs.

@Cebeerre can you please close this issue?

[Zixim]

Does having 2 cars in the integration double the api calls ?

[rikroe]

Does having 2 cars in the integration double the api calls ?

Yes, but you will have to try. It could be that every API endpoint (i.e. One for each vehicle) has its own quota.

[Zixim]

just to be 100% certain : manually updating sensor.ix3_m_sport_charging_status won't update the entities for other car, right ?

[rikroe]

No, it will always update the complete integration config entry, i.e. the full account with all vehicles.

[rikroe]

just to be 100% certain : manually updating sensor.ix3_m_sport_charging_status won't update the entities for other car, right ?

If you would like this feature, please create a separate issue. Needs changes both in the library an HA but should be possible.

[Zixim]

I'm not going to request this feature, but wanted to mention it in this issue since it does have an impact on the number of api calls, increasing the risk of exceeding BMW's quota.

[Cebeerre]

Closing the issue as per @rikroe post.

Thanks !