ViCare API rate limit exceeded with default scan_interval

[bruecktech]

The problem

I have nothing else running against the ViCare API and I'm using the default scan_interval of 60 seconds and still I get throttled every day. This message shows up in the logs:

Vicare API rate limit exceeded: (PyViCareRateLimitError(...), 'API rate limit development portal "e58a75c2-eb4a-48ee-94f6-62475281d509" daily limit exceeded. Max 1450 calls in timewindow. Limit reset at 2022-02-22T00:00:03.615000.')

I tried to increase the interval but since the config is not in YAML anymore there seems to be no way to edit this after the integration was created or am I missing something?

What version of Home Assistant Core has the issue?

2022.2.9

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

vicare

Link to integration documentation on our website

https://www.home-assistant.io/integrations/vicare/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[probot-home-assistant[bot]]

vicare documentation vicare source _{^{(message by IssueLinks)}}

[probot-home-assistant[bot]]

Hey there @oischinger, mind taking a look at this issue as it has been labeled with an integration (vicare) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[oischinger]

I don't know why you exceeded the limits. maybe you used the app more heavily or have some other scripts using pyVicare? For me it works well. I described Viessman's current API limits here: https://www.home-assistant.io/integrations/vicare/#viessmann-api-limits

Unfortunately there is no way to configure the interval anymore. That's actually something Home Assistant does not accept anymore. If you must have more control over the scheduling you can disable automatic updates for each integration and schedule it's updates manually.

[bruecktech]

As I wrote I'm not aware of anything else calling the API. I even uninstalled the app from my phone. The documentation still says this:

It is therefore advised to adjust the scan_interval to your usage scenario

If this is not supported anymore, can we then update this documentation to contain the steps to do the manual scheduling? I had 120 seconds configured before and it was working fine

[8xiom]

Same here, I'm afraid. Had it at 150 sec. scan interval, worked fine. Now I get the API limit exceeded warning every 24 hours as it seems. I must admit that I've set up InfluxDB recently in order to plot the temperature and heat pump compressor data in Grafana. My expectation, though, is that this should not trigger additional or more frequent API calls. I would be glad if the scan_interval option comes back.

Anyway, thanks for the great work, @oischinger! Keep it going, please.

[Ferdi8819]

Same here. I'm using the vicare app multiple times a day. But this was no problem in the past. Additionaly i would expect that the limit would be reached at the end of the day a littel bit earlier instead of 4 pm.

[Fabenissimo]

I can't get the integration to work anymore. The app works fine, every time of the day. If I try to readd the integration to HA I get this error while setting up the integration via the UI: Translation Error: The intl string context variable "name" was not provided to the string "{name}"

Once I started looking into the logs I could see it was complaining about the API-limit, too. So what are the possibilities? Remove the integration from HA for 24 hours and retry to set it up? How has everyone else tired to solve the issue?

[Tho85]

It looks like Vicare had some kind of server-side issue this afternoon, and the integration doesn't cope too well with that. I've added a (non-debug) log here, for further investigation: https://blog.tho.ms/home-assistant-filtered.log.txt

The Vicare server started to send some internal server errors at rount 12:54:00 today, and continued to send 404 errors after that for at least one of the API routes. The integration then tried to fetch that route again and again, and it seems these calls count towards the API limit. About 30 minutes and 1000 API calls later, the API call limit has been depleted.

Maybe you could add some exponential back-off to avoid these kind of failures?

[8xiom]

I can't get the integration to work anymore. The app works fine, every time of the day. If I try to readd the integration to HA I get this error while setting up the integration via the UI: Translation Error: The intl string context variable "name" was not provided to the string "{name}"

Once I started looking into the logs I could see it was complaining about the API-limit, too. So what are the possibilities? Remove the integration from HA for 24 hours and retry to set it up? How has everyone else tired to solve the issue?

Yes, in the following way - which worked for me. (Please use at your own risk.) From the log you should be able to see a date-time stamp which gives you the point in time from which on the API is accessible again. I then deleted the integration, restarted HA, re-added the integration (unfortunatley you need to input all parameters like API key again but at least you don't need a new API key), restarted HA once more and it worked again. Without the restarts the translation error persisted. Might be sufficient to just delete the browser cache, didn't try that. Furthermore, I switched to a manual update of the ViCare sensors and data via the HA update_entity service. Works fine for me for two days now.

[Tho85]

So what are the possibilities? Remove the integration from HA for 24 hours and retry to set it up? How has everyone else tired to solve the issue?

A simple restart after the API limit has been reset (i.e. midnight today) worked for me, no need to remove or reconfigure the integration.

[gomble]

Is it possible to change the scan_intervall in the integration page ? I did not found it.

[8xiom]

Is it possible to change the scan_intervall in the integration page ? I did not found it.

It's gone, you cannot change it via the configuration.yaml anymore. I've deactivated the automatic sensor update of this integration and set up a time triggered manual activation every 4 minutes via the HA update_entitiy service.

But: Checking with the API acitvitiy bar at developer.viessmann.com, it seems that there is still an API call every minute. This was certainly not the case in January, at least on my side. Any ideas?

[tonka3000]

@oischinger I'm also worried about the fixed 60 seconds scan interval since 2022.2. Would it not be easier to change the fix scan interval to 120 seconds per default and let user use the manual way to make more frequent updates. That way most user wont get into "troubles".

The reason for 120 seconds as default:

The 24h limit is 1450 according to your docs (I was not able to find the numbers on developer.viessmann.com, only in the forum). If the limit of 1450 is really fix, it is really easy to hit the rate limiting. 60*24 = 1440 (60 calls per hour), so only 10 calls left. Make the default 120 seconds should lead to 30*24=720 (30 calls per hours), which gives 730 calls left per day for "user activity" or automations.

scan interval	calls left (when no other interaction)
60	10
120	730

@oischinger Thanks for your great work, really appreciate it.

[oischinger]

Hi eveyone, sorry for not responding earlier. I have to admit that I myself never exceeded the limits except for cases where Viessmann servers seemed to have issues. Nevertheless you are right @tonka3000 . The current update rate leaves little room for any other API use.

Unfortunately the ability to choose the scan_interval yourself was removed by request of the HA Core maintainers. They expect us to disable the automatic update on the integration's system options and create an automation which triggers the update. This is not very userfriendly of course.

So I think I would follow your suggestion to set it to 120 sec. Unless anyone else has a better idea?

[adorobis]

730 calls left per day for "user activity" or automations.

I think it is quite a lot, I would opt for some "middle ground" e.g. 90sec intervals or even lower e.g. 70 sec (still over 200 calls for user and automations). Too long intervals will mean that some events won't even get noticed by the integration e.g. burner starts etc.

[oischinger]

I guess people's use cases differ a lot.

From my perspective HA should be the only consumer of the API which is why I would agree that it's not necessary for anyone to have a couple of hundred API calls left.

What I fail to understand why some people in this thread seem to exceed the API limits like @8xiom seem to hit the API limit earlier than others. This scenario cannot be solved by setting the update rate to 120s so it will require updating via an automation

[bruecktech]

Can we maybe add some debug logging to check if more API calls than expected are happening? I have configured it for the vicare component like this but it doesn't seem to log each call.

logger:
  default: warning
  logs:
    homeassistant.components.vicare: debug

I have my polling automation configured to 180 seconds now and at 1am I still have rate-limit errors in the logs occasionally. Looking at the API usage stats it's only HA showing up, so there is no other API consumer.

[oischinger]

Not easily because the rate limit is implemented in the pyvicare library and transparent to the HA integration. But you can check the amount of calls in the Viessmann api control panel I believe

[bruecktech]

Yes I checked it there, but it only shows them with a monthly resolution, so not really useful. How is this working with the manual scheduling when the update_entity service is called? Isn't there a method called synchronously to update the data? Sorry I'm not very familiar with the programming model in HA.

[oischinger]

When creating the pyvicare API you supply a time range as a parameter.

Whenever you retrieve a value from the API you get a value that is between 0 and whatever seconds you defined old

[8xiom]

Update from my side: The number of API calls per hour mysteriously came down to the expected level of 15 (since I manually trigger a sensor update every 4 minutes). I've been monitoring that for the last four days. On March 12th and 13th there were more than 60 API calls p.h. which let me run into Viessmann's API limit. On the 14th the picture was mixed, since yesterday (15th) it is around 15 calls p.h. I changed nothing regarding the Viessmann integration or configuration. Some other updates on HAOS, yes, I did that.

I will now set the trigger back to every 2 minutes to see what happens.

BTW, I checked my Adguard Home log for the viessmann.com domain and could see that there were DNS requests to api.viessmann.com every minute or so plus my automated sensor updates. That corresponded to the approx. 70 - 75 API calls p.h. I saw until March 14th.

[tonka3000]

Hey @oischinger ,

Any plans to change the frequency?

[oischinger]

@tonka3000 I'm still in doubt what is the right limit, especially since Viessmann recently introduced "premium" service models with more API calls.

I would like to document the option to automate a custom polling as a first step: https://github.com/home-assistant/home-assistant.io/pull/22351

[8xiom]

@tonka3000 I'm still in doubt what is the right limit, especially since Viessmann recently introduced "premium" service models with more API calls.

I would like to document the option to automate a custom polling as a first step: home-assistant/home-assistant.io#22351

Might be the best way. No programming effort on your side, and everyone can relatively simple set up a feasible customized sensor update as needed.

[pbox]

It works for me 👍

I disabled polling and created an automation for the entities I need updated :

alias: Update Viessman Device
description: ''
trigger:
  - platform: time_pattern
    minutes: /4
condition: []
action:
  - service: homeassistant.update_entity
    data: {}
    target:
      entity_id:
        - climate.vicare_heating
        - binary_sensor.vicare_burner_active
        - sensor.vicare_outside_temperature
mode: restart

[adorobis]

Has there been any change in the integration? As of version 2022.7.0 I'm hitting the API rate limit, before that I was not getting this error for many months.

[adorobis]

Just checked the statistics on the Viessmann developer portal, it has been quite consistent in the last months but July is indeed much higher: To me it looks like in the first 6 days of July it used less than 1100 calls per day and after upgrade it is hitting the limit already around 5:00 PM. Any ideas?

Edit: Just looked in the DB and it seems that the problem has started a bit earlier. Until 3 July all worked ok and since 4th July I'm hitting the API rate limit in the afternoon, even before upgrading HA to 2022.7.0

[adorobis]

I've also checked the statistics from recorder. They look the following way. Seems that there are no more calls in the last few days compared to previous 2 weeks. So must be something on Viessmann API side? Date State_count 2022-06-24 1254 2022-06-25 965 2022-06-26 881 2022-06-27 1254 2022-06-28 1177 2022-06-29 1176 2022-06-30 1030 2022-07-01 927 2022-07-02 725 2022-07-03 823 2022-07-04 841 2022-07-05 759 2022-07-06 715 2022-07-07 797 2022-07-08 816

[oischinger]

For me the numbers on the dev portals didn't really change in July. Everything seems consistent. Also I don't see any change in recent HA that would lead to more frequent requests.

[adorobis]

So what could go wrong? Today the same thing, just a bit later. Additionally now I'm getting the error in the log 14 times in every minute. Not sure really what to do apart from disabling the auto update...

[oischinger]

Are you using the custom integration or the official one? Can you confirm that the rate limit isn't exceeded with the previous HA version?

Did you setup an automation for manual polling?

[adorobis]

I'm using the standard component since a few months already as it has all the features I need :) The rate limit started to be exceeded on 4th of July, so 2 days before upgrade to 2022.7.0 Before that I was not exceeding the rate limit at all. So far I had it with automated polling, yesterday I've disabled it and set up automation to poll it every 2 minutes. I have not changed anything and suddenly it started to hit the rate limit...

Here I have a question: do I need to specify each entity I want to be updated or will it update all of them when only selecting climate entity? Edit: just got the answer as one of the entities was not updated as I've missed it in the automation.

[arekchy]

In my case API rate is always exceeded after my gas heater failure (I have some problems with heater and it get locked from time to time). When my heater stops working API calls are running out pretty fast - didn't have time to find more info, but wanted to share - maybe there is a way to decrease calls frequency in this scenario.

[AnHardt]

I do see about the same. If the connection between heating and viessmann-server breaks for any reason the call limit is exceeded within minutes. Even if automatic update cycle is off and manual (automation) updates are made only every 5 minutes.

[tonka3000]

Could be the same in my case as well. I was already confused why I have many "rate limiting exceeds" problems in the last couple of weeks. Previously I didn't have this kind of problem that often. Sometimes my vicare wifi device is not reachable and rate limiting problems occour pretty close to such "events" .

[AnHardt]

For me it looks a bit like if you get a error 502 "DEVICE_COMMUNICATION_ERROR" a retry is made for every single sensor instead of accepting the missing result for all sensors for this time and retrying at the next scheduled time. That way a failure at one time counts 'number of sensor' times for the amount of requests. For me the integration recovers automatically at about 0:00 UTC

[awirtz]

I see the same thing as people above and agree with @AnHardt that this might be the cause for the error. Same message in my log files and quickly hitting the limit during the day. I've switched off automatic updates for now and switched over to use the automation as shown by @pbox with just the entities that I need.

[pbox]

I'm happy with the config, I started as of this morning to update it every 2 minutes (~720/day) instead of every 4 minutes (~360/day).

Here's the api usage so far (every 4 minutes) :

[adorobis]

Those numbers are way below the limit. My stats look the following. In August I've changed to refresh it every 2 minutes as I was getting locker. But in October I'm back to default 1 minute interval and no locks at all. Viessmann must have done something wrongly on their end.

[oischinger]

For me it looks a bit like if you get a error 502 "DEVICE_COMMUNICATION_ERROR" a retry is made for every single sensor instead of accepting the missing result for all sensors for this time and retrying at the next scheduled time. That way a failure at one time counts 'number of sensor' times for the amount of requests. For me the integration recovers automatically at about 0:00 UTC

@woehrl01 what do you think? Looking at the implementation of PyVicare's PyViCareCachedService this could actually be the case. We could be hammering the API whenever we get DEVICE_COMMUNICATION_ERROR.

[sebabar]

Hi, i can observe the same issue. Most days it works ok and no limit is exceeded. From time to time, it fails. Today afternoon (as checked in logs) I started to receive various strange errors (Unable to decode data from ViCare server, Missing 'data' property when fetching data etc.). Some time later errors 404, 502, and 500 started to appear. Logs got flooded with errors and after 8pm I got "limit exceeded" error. It really looks like Viessmann's server issues cause integration going mad trying to get data despite the failures, and this leads to uncontrolled requests flood.

Regards, Sebastian

[factor4]

Hi! I think so too. Yesterday I saw a maintainence message in the ViCare-App and after a while I was blocked due "limit exceeded".

[joris628]

Hi! I think so too. Yesterday I saw a maintainence message in the ViCare-App and after a while I was blocked due "limit exceeded".

Same thing happened to me yesterday

[jzakotnik]

I can confirm that I get the same problem with the API limit after a day. Since I turn off the WLAN at night, it is likely that the ViCare box cannot connect to the API anymore and therefore the home assistant retries too many times. I will try to validate this, but it would indicate that the problem is with the "device communicate error".

[woehrl01]

@jzakotnik @AnHardt I just published a fix. Updating to the latest version of the base library should be sufficient.

[AnHardt]

No Problems of this kind anymore.

[jzakotnik]

Many thanks, also helped here!

[zoogian]

It works for me 👍

I disabled polling and created an automation for the entities I need updated :

alias: Update Viessman Device
description: ''
trigger:
  - platform: time_pattern
    minutes: /4
condition: []
action:
  - service: homeassistant.update_entity
    data: {}
    target:
      entity_id:
        - climate.vicare_heating
        - binary_sensor.vicare_burner_active
        - sensor.vicare_outside_temperature
mode: restart

hello

New to this integration and fairly new to HA, so please be kind... :).

this code for update api call, it is just in automations.yaml ?

OR is there any other guide to set update to not so frequent? Use the app a lot so dont whant to risk hitting the limit.