Closed Cebeerre closed 2 years ago
bmw_connected_drive documentation bmw_connected_drive source (message by IssueLinks)
Hey there @gerard33, @rikroe, mind taking a look at this issue as it has been labeled with an integration (bmw_connected_drive
) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)
Can confirm, thanks for the report. Could be a temporary configuration thing at BMW or a very deliberately one. I'll see if I find some time to debug this tomorrow.
Same here. MyBMW app isn't updating vehicle status too.
MyBMW app was updated 12 hours ago so probably (hopefully) not related to HA integration?
Same here after updating to 2022.9.5
Same here - still on 2022.9.4 so probably not related to the new version...
It seems as if BMW pulled on too tight quotas - it worked for almost the whole night for me (after a restart of HA) and then just this morning it had some issues (that self-healed).
It might not even be related to the 5 minute polling intervall, but rather if we do our requests too quickly:
2022-09-20 08:06:52.526 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK"
2022-09-20 08:06:52.901 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK"
2022-09-20 08:06:53.454 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state "HTTP/1.1 200 OK"
2022-09-20 08:06:53.458 DEBUG (MainThread) [custom_components.bmw_connected_drive.coordinator] Finished fetching bmw_connected_drive-ACCOUNT data in 2.457 seconds (success: True)
2022-09-20 08:11:53.893 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK"
2022-09-20 08:11:54.309 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK"
2022-09-20 08:11:54.358 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state "HTTP/1.1 403 Quota Exceeded"
2022-09-20 08:11:54.361 ERROR (MainThread) [custom_components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-ACCOUNT data: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state'
2022-09-20 08:11:54.362 DEBUG (MainThread) [custom_components.bmw_connected_drive.coordinator] Finished fetching bmw_connected_drive-ACCOUNT data in 1.361 seconds (success: False)
2022-09-20 08:16:54.231 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 403 Quota Exceeded"
2022-09-20 08:16:54.233 DEBUG (MainThread) [custom_components.bmw_connected_drive.coordinator] Finished fetching bmw_connected_drive-ACCOUNT data in 0.233 seconds (success: False)
2022-09-20 08:21:54.360 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 403 Quota Exceeded"
2022-09-20 08:21:54.362 DEBUG (MainThread) [custom_components.bmw_connected_drive.coordinator] Finished fetching bmw_connected_drive-ACCOUNT data in 0.361 seconds (success: False)
2022-09-20 08:26:54.744 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK"
2022-09-20 08:26:55.065 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK"
2022-09-20 08:26:55.537 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state "HTTP/1.1 200 OK"
2022-09-20 08:26:55.541 INFO (MainThread) [custom_components.bmw_connected_drive.coordinator] Fetching bmw_connected_drive-ACCOUNT data recovered
At 08:11:54.358
there was a request only 0.049
seconds after the last, while it is usually rather around 0.5
seconds. But it self-healed at least...
I'll continue to monitor this.
It seems as if BMW pulled on too tight quotas - it worked for almost the whole night for me (after a restart of HA) and then just this morning it had some issues (that self-healed).
It might not even be related to the 5 minute polling intervall, but rather if we do our requests too quickly:
2022-09-20 08:06:52.526 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK" 2022-09-20 08:06:52.901 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK" 2022-09-20 08:06:53.454 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state "HTTP/1.1 200 OK" 2022-09-20 08:06:53.458 DEBUG (MainThread) [custom_components.bmw_connected_drive.coordinator] Finished fetching bmw_connected_drive-ACCOUNT data in 2.457 seconds (success: True) 2022-09-20 08:11:53.893 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK" 2022-09-20 08:11:54.309 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK" 2022-09-20 08:11:54.358 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state "HTTP/1.1 403 Quota Exceeded" 2022-09-20 08:11:54.361 ERROR (MainThread) [custom_components.bmw_connected_drive.coordinator] Error fetching bmw_connected_drive-ACCOUNT data: Error communicating with BMW API: Client error '403 Quota Exceeded' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state' 2022-09-20 08:11:54.362 DEBUG (MainThread) [custom_components.bmw_connected_drive.coordinator] Finished fetching bmw_connected_drive-ACCOUNT data in 1.361 seconds (success: False) 2022-09-20 08:16:54.231 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 403 Quota Exceeded" 2022-09-20 08:16:54.233 DEBUG (MainThread) [custom_components.bmw_connected_drive.coordinator] Finished fetching bmw_connected_drive-ACCOUNT data in 0.233 seconds (success: False) 2022-09-20 08:21:54.360 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 403 Quota Exceeded" 2022-09-20 08:21:54.362 DEBUG (MainThread) [custom_components.bmw_connected_drive.coordinator] Finished fetching bmw_connected_drive-ACCOUNT data in 0.361 seconds (success: False) 2022-09-20 08:26:54.744 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK" 2022-09-20 08:26:55.065 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK" 2022-09-20 08:26:55.537 DEBUG (MainThread) [httpx._client] HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state "HTTP/1.1 200 OK" 2022-09-20 08:26:55.541 INFO (MainThread) [custom_components.bmw_connected_drive.coordinator] Fetching bmw_connected_drive-ACCOUNT data recovered
At
08:11:54.358
there was a request only0.049
seconds after the last, while it is usually rather around0.5
seconds. But it self-healed at least...I'll continue to monitor this.
it's indeed quite random, as it works for maybe two or three hours then maybe doesn't work for two hours (or 5 minutes), then self-heals ... so, yes, probably this has something to do more with a rate limit than with a total amount of queries ...
I'll try to do some tests later today ...
FWIW, everything was fine for me until 3PM ET yesterday.
Same here, if it helps uploading more logs, let me know
Obvisously, everyone is affected... hopefully is a temporary malfunction and not a permanent strict quota policy.
Same problem here. Car is also thrown out of the 'My BMW'-app and it can not be added anymore. I guess that is connected to this problem?
Same Problem here. But the myBMW App is still working.
Same problem, started yesterday. Sending commands from the myBMW app works fine, but the charge state does not update. I´ve opened a ticket with BMW.
Also logging out and in again in the app will yield 0 cars.
perhaps superfluous - but same issues here - worked untill today for me est 1500 hours - then nothing any more... also the quota messages.
Looking over the last ~30hrs of HA entity availability (which I can confirm correlates to my 403 quota issues), it appears my API quota reset promptly at 5PM PST (which is coincidentally midnight UTC). I am guessing we are dealing with a [new??] daily quota. Based on how long my requests were completing successfully (~10 hrs), I am also going to make a guess that a default, 1-car, minimal MyBMW app usage cases that quota to be exceeded by about 3x the allowable rate. As such, I have attempted to reduce my scan interval by 3x to 900s (or once every 15m). I added some notes as to what I did over here. If the above is correctly, it sounds like we will have to wait another 3 hours to test this theory (when the daily quota likely resets).
Looking over the last ~30hrs of HA entity availability (which I can confirm correlates to my 403 quota issues), it appears my API quota reset promptly at 5PM PST (which is coincidentally midnight UTC). I am guessing we are dealing with a [new??] daily quota. Based on how long my requests were completing successfully (~10 hrs), I am also going to make a guess that a default, 1-car, minimal MyBMW app usage cases that quota to be exceeded by about 3x the allowable rate. As such, I have attempted to reduce my scan interval by 3x to 900s (or once every 15m). I added some notes as to what I did over here. If the above is correctly, it sounds like we will have to wait another 3 hours to test this theory (when the daily quota likely resets).
I can't see my vehicle in myBMW app after logout/login. So it's pretty massive bug of bmw's systems rather than "new quota" feature...
Same Problem here. Worked fine till yesterday. MyBMW app also doesnt Update location/status.
The car is back in "My BMW" app since tonight. Data is up to date again in Home assistant also. I had a test version of HA with the BMW connected integration activated, but I have disabled it since yesterday to lower the amount of data requests.
Right. It does seem that it's a daily rate limit quota which reset about 2.5 hrs ago. Reports on [this issue]() seem to corroborate this theory of a UTC midnight reset.
Same here, so it seems that the polling needs to be limited?
Thanks for the debugging guys! I'll implement a higher polling interval (probably 30 minutes to be safe) later today and make it configurable via the integration options.
Today I woke up with everything available on my HA, and 2 hours later it's already unavailable :(
Same here. 6 hours ago I mentioned that all was working again. Now HA has 403 error again. Csr is still available in the BMW app, but last update was 2 hours ago.
It worked for three hours for me today, so that must be 36 requests. If we need to spread those out onto an entire day, I guess we need at least 40 minutes in between the requests. If the app requests count against the same limit as well, we will probably need even more time in between the request - otherwise could we risk not being able to use the app??? Just guessing here :-)
Thanks for the debugging guys! I'll implement a higher polling interval (probably 30 minutes to be safe) later today and make it configurable via the integration options.
This is a good approach. Eventually we'll be able to determinate the quota (calls / day).
It worked for three hours for me today, so that must be 36 requests. If we need to spread those out onto an entire day, I guess we need at least 40 minutes in between the requests. If the app requests count against the same limit as well, we will probably need even more time in between the request - otherwise could we risk not being able to use the app??? Just guessing here :-)
It seems that there is no relation between the app and the quota. For example, yesterday i was getting the quote exceeded message in HA, but the app was fine (i've tried also to send a flash lights command and was executed successfully). How they discriminate the traffic between the app and other requests is still unknown. I'll capture the traffic with wireshark and see if they're using any extra headers or something.
Same here with a very clear pattern:
It starts working at midnight, becomes intermittent at dawn and completely becomes unavailable a few hours later.
SoC in the MyBMW app main screen ceases to work too. In the details, SoC is still available.
Already contacted BMW complaining about the app not working correctly, maybe they will raise the quotas if we all complain.
Another idea that just came to my mind: At each refresh, we are currently calling at least 3 endpoints: 2 for basic vehicle information (one for BMW, one for MINI, e.g. here) and the actual vehicle state which is what is most interesting to us. The MyBMW also always calls both endpoints.
Maybe we can skip reloading the vehicles endpoint at every polling cycle and just call it at first execution, reducing the number of calls by two thirds.
It worked for three hours for me today, so that must be 36 requests. If we need to spread those out onto an entire day, I guess we need at least 40 minutes in between the requests. If the app requests count against the same limit as well, we will probably need even more time in between the request - otherwise could we risk not being able to use the app??? Just guessing here :-)
It seems that there is no relation between the app and the quota. For example, yesterday i was getting the quote exceeded message in HA, but the app was fine (i've tried also to send a flash lights command and was executed successfully). How they discriminate the traffic between the app and other requests is still unknown. I'll capture the traffic with wireshark and see if they're using any extra headers or something.
Mmmm, I am not entirely sure. My app does not work completely now. The SoC is not updated on the main page, but when I go to the details the values are correct. So something seems to stop working, when the quota has been reached 😕
It worked for three hours for me today, so that must be 36 requests. If we need to spread those out onto an entire day, I guess we need at least 40 minutes in between the requests. If the app requests count against the same limit as well, we will probably need even more time in between the request - otherwise could we risk not being able to use the app??? Just guessing here :-)
It seems that there is no relation between the app and the quota. For example, yesterday i was getting the quote exceeded message in HA, but the app was fine (i've tried also to send a flash lights command and was executed successfully). How they discriminate the traffic between the app and other requests is still unknown. I'll capture the traffic with wireshark and see if they're using any extra headers or something.
Mmmm, I am not entirely sure. My app does not work completely now. The SoC is not updated on the main page, but when I go to the details the values are correct. So something seems to stop working, when the quota has been reached 😕
Perhaps because im using the MINI app? In any case, the app is not affected during the last 2 days which i'm getting the quota exceeded errors. Meanwhile i've disabled the automatic polling on BMW integration and using the update_entity service on time pattern automation.
- service: homeassistant.update_entity
data: {}
target:
entity_id:
- device_tracker.cooper_s
- binary_sensor.cooper_s_check_control_messages
- binary_sensor.cooper_s_condition_based_services
- binary_sensor.cooper_s_door_lock_state
- binary_sensor.cooper_s_lids
- sensor.cooper_s_remaining_range_fuel
- sensor.cooper_s_mileage
- binary_sensor.cooper_s_windows
Perhaps because im using the MINI app? In any case, the app is not affected during the last 2 days which i'm getting the quota exceeded errors. Meanwhile i've disabled the automatic polling on BMW integration and using the update_entity service on time pattern automation.
You just need to use update_entity
on one entity, all others will be refreshed automatically.
Perhaps because im using the MINI app? In any case, the app is not affected during the last 2 days which i'm getting the quota exceeded errors. Meanwhile i've disabled the automatic polling on BMW integration and using the update_entity service on time pattern automation.
You just need to use
update_entity
on one entity, all others will be refreshed automatically.
I had a suspicion that it would work like this but I wasn't sure, thanks for confirming!
Disabled my bmw integration yesterday and today I can finally see the car again in the app.
Maybe we can skip reloading the vehicles endpoint at every polling cycle and just call it at first execution, reducing the number of calls by two thirds.
Like that idea
Unfortunately you can only add one time the car in the app, which means you cannot add it to the Home Assistance. Once you did, it will kick you out of the phone app. This is what now happened to me and I seem not to be able to add it again. So I have to remove it from HA. :(
Unfortunately you can only add one time the car in the app, which means you cannot add it to the Home Assistance. Once you did, it will kick you out of the phone app. This is what now happened to me and I seem not to be able to add it again. So I have to remove it from HA. :(
Disable the integration in HA (or at least disable polling in the integration's system options), then you should be able to see your can in the MyBMW app starting tonight 0:00 UTC.
Following a quick discussion with the HA team, disabling the default polling in HA and using automations with a custom trigger of 20 minutes+ (https://github.com/home-assistant/core/issues/78792#issuecomment-1253593396) is the best workaround for now.
With https://github.com/home-assistant/core/pull/78910 merged, the reduction of API calls will be in 2022.9.6.
It is still a five-minute interval, since this is a separate API endpoint which we don't know the quota of. My guess is that 10 minutes should be possible, maybe even 5.
If you are currently blocked due to the quota limit, you'll have to wait until the limit is lifted again for it to work (first two calls after HA start/component reload are going agains the currently limited endpoints and are required on initialization).
In my (non-HA) automation, polling each 15min triggered the quota after 17h.
In my (non-HA) automation, polling each 15min triggered the quota after 17h.
Did you use bimmer_connected==0.10.4
(released yesterday evening) already? This reduces the number of calls to the currently blocking endpoint massively (if you keep the MyBMWAccount object in Python memory and not using the CLI).
@rikroe not yet. Just updated to 0.10.4, will observe.
I have the app installed on 2 phones. Is the maximum amount of polling shared between al devices? So if I poll with 2 phones, doesn't that also lower the amount of polling HA is allowed to do?
I have the app installed on 2 phones. Is the maximum amount of polling shared between al devices? So if I poll with 2 phones, doesn't that also lower the amount of polling HA is allowed to do?
While technically true (as far as I understand the API & quotas), I don't think it using two phones will have that big of an impact. The MyBMW apps only poll the APIs when opened, so you do not create the same amount of calls as we do (or did before 2022.9.6).
I´ve disabled both HA and my smartcharger BMW integration, even changed my password to make sure. Now the only thing polling should be my phone app. It stopped working around 13:30 GMT+2. BMW clearly has some work to do...
I´ve disabled both HA and my smartcharger BMW integration, even changed my password to make sure. Now the only thing polling should be my phone app. It stopped working around 13:30 GMT+2. BMW clearly has some work to do...
Has this been reported to BMW? if so any reference number so we can all report with the same and get it looked at?
I have an open ticket, no reference other than my VIN unfortunately.
@rikroe I'm new to the HA ecosystem as well as this integration. Trying to catch up, I read through bimmer_connected and its sibling solutions (linked from the README) - it looks like we all are using the My BMW API that powers that android/iOS apps. Nowhere can I find this as officially sanctioned, documented, or supported by BMW. Is that the case? Also curious if the My BMW EULA explicitly prohibits such use.
If yes to either above, I only bring it up to help set expectations for everyone above who may be upset / creating tickets with BMW. In this case, it seems that BMW is acting in their best interests to protect and manage the official, intended use case for this API (ie, they are not expecting or supporting users making thousands of API calls each day) while keeping costs down. My base assumption would be that we as users of this integration should expect this to work on an as-is basis, and know that the HA integration may inadvertently break the official MyBMW app usage at any time (as is the case in this issue); conversely, BMW may change the undocumented API at any time (including model layout and rate limits) which can/has broken the HA integration.
From what I did find, it seems like they expect customers (or rather solution providers) to use their sanctioned CarData API.
@jprouty this is a perfect summary. The library (and all its predecessors) are not officially endorsed/sanctioned/supported by BMW and is actually reverse-engineered by capturing the HTTP calls between the app on the phone and the BMW servers.
The CarData API you mentioned could be used e.g. by a solution provider that in turn passes down the costs to the customers (up to €5/user/month) and has some restrictions (e.g. no possibility to execute services on the vehicle).
Typically, the HA user is the owner of the car, thus already paying for BMW Connected Drive, trying to retrieve his own data...
The problem
Seems that BMW has limited the amount of queries you can do to their API, as it's now throwing a 403 with a "Quote Exceeded" message making all the sensors unavailable.
What version of Home Assistant Core has the issue?
2022.9.5
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant Container
Integration causing the issue
bmw_connected_drive
Link to integration documentation on our website
https://www.home-assistant.io/integrations/bmw_connected_drive/
Diagnostics information
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Additional information
No response