Closed hmmbob closed 1 year ago
Hey there @bachya, mind taking a look at this issue as it has been labeled with an integration (openuv
) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)
openuv documentation openuv source (message by IssueLinks)
Since OpenUV doesn't have automatic API polling, it'd be useful to see the automation you have for updating data.
Sure:
- id: update_openuv
alias: "Update OpenUV"
trigger:
- platform: time_pattern
minutes: "/25"
- platform: homeassistant
event: start
condition:
- condition: sun
after: sunrise
after_offset: "+00:30:00"
before: sunset
before_offset: "-00:30:00"
action:
service: homeassistant.update_entity
target:
entity_id:
- binary_sensor.openuv_protection_window
- sensor.openuv_current_uv_index
But while this is the reason I ran out of API credits (due to restarts and "home assistant start" as trigger), the issue I'm trying to describe comes after you run out of API credits: then the component keeps trying over and over and over again to get the data (until the API finally accepts a request because a new day has started).
The log above is from just 1hr....
I'm confused why that's an issue? Wouldn't you expect the integration to keep trying until it can reestablish a connection?
The API response doesn't give us any clarity on a date or time when the rate limiting will expire, so our only option is to keep trying.
True, but every 1min20? Seems a bit much.
Maybe some increasing timeout our so? Or Log the error when receiving a 403 (as that apparently means "out of API credits" for openuv), but don't retry automatically and wait for the next manual update (as this is, as you rightly pointed out, a component without any default polling).
Api credits seem to reset at 00:00 GMT by the looks of my logs
I think I see what's happening.
When OpenUV is first loaded, we disable the retry mechanism used by the underlying library (pyopenuv
) and attempt to grab data for the first time—any failure will initiate a retried loading of the entire integration (orchestrated by HASS itself):
As you can see from that second snippet, we watch for two different error conditions:
In your case, the first error type is what's occurring.
If this were to happen after the integration loads, we would increment a counter and watch for subsequent HTTP 403
s (in short, OpenUV can return a 403
because of an invalid key or an underlying service issue; more background: https://github.com/home-assistant/core/pull/79691). However, in this case, the integration hasn't loaded; Home Assistant itself will retry loading, which unfortunately bypasses our counter (and prevents us from stopping checks).
All this said, it looks like OpenUV has updated their API with some new calls (including one to determine if the service is down). Let me take a look at updating pyopenuv
and then return here.
Wow, very detailed explanation!
Thanks much anyway for maintaining this integration. While currently it's only rain around here, we use it quite a bit during the nice seasons!
@hmmbob Okay, I have a route forward: https://github.com/home-assistant/core/pull/85573. It's worth quickly reading over the idiosyncrasy:
There is one quirk to keep in mind. In OpenUV, an
HTTP 403
response indicates one of two conditions:
- An invalid API key
- An API key whose daily/monthly limit is reached
Unfortunately, we can't deduce which is which when we receive an
HTTP 403
(there is an API endpoint to determine how many calls a user has made per day/month, but there is no way to know what that particular user's call limit is 🤦🏻). So, we follow this strategy:
- Any
HTTP 403
will trigger a re-auth flow.- In the case of an overrun API call limit, once the limit expires and the coordinator successfully retrieves data, any existing re-auth flow is canceled.
TL;DR: if you have a key whose limit is reached, you'll get a notice about an invalid API key (and an invitation to enter a new one). If you ignore that until the next time your automation runs—after the key limit is reset—that re-auth notice will automatically go away. I'll be opening a docs PR to ensure this is documented.
Oh, this is great!
About:
Unfortunately, we can't deduce which is which when we receive an HTTP 403 (there is an API endpoint to determine how many calls a user has made per day/month, but there is no way to know what that particular user's call limit is 🤦🏻).
The API provides you either 50 (free plan) or 15000 (paid) credits a day - is that something to work with?
As in: if i get a 403 AND I have made exactly 50 calls according to the API, then it is safe to assume I ran into my limit of the free plan. Any other value would be inconclusive.
Just a thought. Already very thankful for your great work here!
The API provides you either 50 (free plan) or 15000 (paid) credits a day - is that something to work with?
As in: if i get a 403 AND I have made exactly 50 calls according to the API, then it is safe to assume I ran into my limit of the free plan. Any other value would be inconclusive.
Thanks for the idea, @hmmbob. That could still pose a situation where an API has become invalid and we mistakenly think it's just at its limit. That said, it's an interesting line of though—I'll keep investigating!
Came across another piece of code that solves this quite elegantly:
Something like this could work. I guess it's safe to assume max 50 daily API calls (I'm not sure why someone would pay $15 for 15000 daily API calls - or IF you want to include this edge case this could be a configuration setting? Linke a checkbox in the settings?), and if we use sunrise/sunset times smartly we can calculate how many minutes should be between updates. Sun info is encompassed in the /api/uv
API endpoint, so it's pretty much self contained anyways (no dependency on sun.sun
)
I was playing with the dashboard of OpenUV, ~and it appears that calls to /api/stat
are not counted towards the API credit.~ That's interesting, because. it contains the amount of API calls made already.
IF the API still allows us to call this endpoint when we exceeded the amount of calls, we have a stable way to determine if 'API calls exceeded' is the reason for error.
sample data:
{
"result": {
"cost_last_month": 0,
"cost_month": 0.065,
"cost_today": 0,
"cost_yesterday": 0,
"requests_last_month": 1398,
"requests_month": 310,
"requests_today": 46,
"requests_yesterday": 34
}
}
Edit: playing around a bit more revealed two things:
{ "error": "Daily API quota exceeded. Add billing details to get 15000 reqs/day or contact support@openuv.io to upgrade to Unlimited Plan."}
edit2:
bob@debian:~/docker/smarthome/homeassistant$ curl -X GET 'https://api.openuv.io/api/v1/uv?lat=51.0&lng=5.0&alt=100&dt=' -H 'x-access-token: 1234567890'
{"error":"User with API Key not found"}
So, by looking to the respons JSON we can determine what is the issue:
{ "error": "Daily API quota exceeded. Add billing details to get 15000 reqs/day or contact support@openuv.io to upgrade to Unlimited Plan."}
or
{"error":"User with API Key not found"}
Tagging you because I don't know if you still receive notifications for this thread; @bachya
Great detective work, @hmmbob! I'll definitely give this a look.
The problem
I was tweaking my HA setup yesterday, with a lot of restarts of the system and because of that - I ran out of API calls (as the sensors get updated on a restart) - which is fine/understood.
The component doesn't handle this well - it will keep trying to request the data every minute or so until the new API-day starts.
What version of Home Assistant Core has the issue?
core-2023.1.1
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant Container
Integration causing the issue
OpenUV
Link to integration documentation on our website
https://www.home-assistant.io/integrations/openuv/
Diagnostics information
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
And this is just a snippet...... (LatLong redacted but are fine).
Additional information
No response