Hitting the API limit

[cloneofghosts]

I had the same issue. The odd part is that I've had this integration set up exactly the same way since December, and this is the first month I've run into the API limit.

Originally posted by @mermelmadness in https://github.com/Pirate-Weather/pirate-weather-ha/issues/218#issuecomment-2069435253

[cloneofghosts]

Hi @mermelmadness I created this issue for you from the comment. You're saying that you hit the API key limit for the month when you didn't before?

[alexander0042]

Thanks @cloneofghosts for organizing this correctly! Weird glitch @mermelmadness, and not sure what went happened here. If you let me know the email you signed up with or send me an e-mail at api@pirateweather.net, I'll up the limit for the rest of the month.

[mermelmadness]

Thanks for this.

Back in November this forum helped me re-setup the integration using the new GET weather protocol used in HA (Issue #137). Ever since then the configuration has remain unchanged on my end. Yet in April I hit the API limit.

The e-mail I used is mermelmadness@gmail.com

These are my current versions:

IIRC, there is a way to up the API limit per month for a small fee. Can you remind me what that monthly fee is? I am considering paying for it since I use this many times daily.

[cloneofghosts]

I'm going to assume you've already queried the browser to confirm that you've hit the API limit? Since you haven't hit the limit before you have your update interval low enough to not hit the limit? You could also try to regenerate your API key to see if it solves things though you'd have to setup the integration again if you want to change your API key.

Here is a list of all available plans available.

[mermelmadness]

Correct

I think I see what happened. The plans changed. The free version used to allow 20,000 calls per month. I know this because I set my interval to be 00:02:15 because that would make 19,840 calls in a 31-day month.

Now I'll set my interval to 00:04:28 which will make 9,995 calls per 31-day month.

[cloneofghosts]

That makes sense why you hit the limit this month when you weren't before. @alexander0042 must have introduced a new tier with the same amount of calls for $2 and reduced the number of calls in the free tier.

[alexander0042]

Hi, thanks for reaching out and opening this issue, I haven't changed anything recently, so the timing for this is sort of strange; and I'm wondering when you started seeing these errors show up- I'm seeing an increase in 429s (quota) errors starting April 15, so wondering if you aligned with that?

[mermelmadness]

Not exactly. Looks like the entities stopped populating data on April 21st

[mermelmadness]

@alexander0042 The API plans definitely changed, though. I distinctly remember setting my interval to come just under the free limit, so it was 20,000 at one point. Maybe AWS changed something under the hood?

[mermelmadness]

I got my entities back. Did something change? Wasn't expecting any data until May 1st.

[cloneofghosts]

Maybe @alexander0042 upped your quota for this month to get things working again?

[mermelmadness]

That was my guess originally, but the entities haven't updated in 6 hours, so not sure.

[mermelmadness]

And now it's unavailable again.

I'm ok waiting for May 1st for the quota to reset.

[cloneofghosts]

Do you get the message about your API quota being reached when you visit the browser or do you get a different error? Anything in the logs?

[alexander0042]

Yea, this is definitely a weird one! Is there any chance you signed up using a different e-mail address? I can't even find that one in the system

[alexander0042]

Wait, found it! I might know what happened here, give me 5 min

[alexander0042]

Update, I've got no idea here, since everything looks ok on my end. You have one of the OG legacy keys, so actually set at a 25k cap, since I started very untrustworthy of my quota system. It's showing it entirely used, but I don't have enough logging to see exactly when.

I upped the limit to 50k and enabled logging, so I'll be able to see exactly what calls are made for now. This will give a nice view of what's happening going forward, so should get to the root cause

[mermelmadness]

Do you get the message about your API quota being reached when you visit the browser or do you get a different error? Anything in the logs?

In the browser, yes. I didn't really check the logs since it seemed like my issue was the same as someone else's.

[mermelmadness]

Update, I've got no idea here, since everything looks ok on my end. You have one of the OG legacy keys, so actually set at a 25k cap, since I started very untrustworthy of my quota system. It's showing it entirely used, but I don't have enough logging to see exactly when.

I upped the limit to 50k and enabled logging, so I'll be able to see exactly what calls are made for now. This will give a nice view of what's happening going forward, so should get to the root cause

Thank you. I look forward to see if any results show themselves. I can confirm the integration is working again on my end. I'll leave my interval as such for now:

  - platform: pirateweather
    api_key: <APIKEY>
    scan_interval: '00:04:28'

[cloneofghosts]

Since everything seems to be sorted I'm going to close this for now. If you encounter more issues you can re-open this issue or create a new one.

[mermelmadness]

Dang, was about to sign off of work and post that this issue could be closed. Beat me to it by 2 minutes.

Thanks for all your help. Everything seems to be working as expected.

[alexander0042]

Ok, since it's been a couple days and I've got some new data, I think we should re-open this, since it might be a larger issue with the HA integration. The simple reason you hit the limit is because I'm seeing calls every ~2 minutes from your API key, but th harder one is why:

First and foremost, can you double check you only have one version of the integration instated but looking in your entities page? And do you know if it was installed via the yaml file or the UI? I'm wondering if somewhere along the line HA imported the YAML again, which is why you're getting 2x the calls that should be happening

[mermelmadness]

I have 57 entities for the PirateWeather integration, but no duplicates. Could it be it's going twice because I have two entries, one for weather and one for forecast?

Oddly, 3 entries are showing up:

Originally it was YAML based, but ever since the new GUI option was implemented I re-set it up that way to take advantage of additional data.

I still have a YAML entry. Should I remove that?

[alexander0042]

Yea, that must be it. The entities and forecast both use the same data update coordinator, so share API calls, but I suspect that rogue "8 entities" one is the issue here. Can you try deleting that, as well as (or commenting out) your YAML, and we'll see what that does.

[mermelmadness]

That makes sense since the entities match the YAML:

• platform: pirateweather api_key: scan_interval: '00:04:28' monitored_conditions:
  • temperature
  • precip_probability
  • precip_type
  • humidity
  • cloud_cover
  • nearest_storm_distance
  • precip_intensity
  • wind_speed

Commenting out and restarting...

[mermelmadness]

Had to manually delete the extra entry, but it did not automatically reappear.

Please monitor my API calls and see if the interval has changed....though I think I read that in order to change the interval I need to redo the integration and I'm not prepared to do that right now as I have several automations that use this integration.

[jhemak]

Yea, that must be it. The entities and forecast both use the same data update coordinator, so share API calls, but I suspect that rogue "8 entities" one is the issue here. Can you try deleting that, as well as (or commenting out) your YAML, and we'll see what that does.

Hi @alexander0042 - could you please clarify this a bit more? I previously had my integration set up similarly to @mermelmadness with one instance for Entities and another for Weather. Honestly, I can't recall why I had done that. But when I ran into an API Limit issue this month, @cloneofghosts suggested (#218) consolidating to a single instance of the integration. My conclusion from that was that the former configuration was causing twice the API calls of the latter. Is that how it works?

[cloneofghosts]

Hi @mermelmadness and @alexander0042 just checking in to see if this issue has been solved and can be closed. The last update was just over a month ago and there has been no updates since which leads me to believe this has been solved.

[mermelmadness]

Hi @mermelmadness and @alexander0042 just checking in to see if this issue has been solved and can be closed. The last update was just over a month ago and there has been no updates since which leads me to believe this has been solved.

My issue is resolved. I cannot speak for everyone who chimed in on this issue.

[cloneofghosts]

I created a separate issue for the comment left by @jhemak so this can be closed. Thanks.

[catchdave]

@alexander0042 - I think I am having a similar issue. My entities are all reporting errors with "429 - too many requests". The only change I did this month was upgrade Home Assistant and other HA modules.

Can I DM you my API key or email to look up what's going on? My scan interval when it was YAML based was 5minutes, I guess it's the same when it was migrated to the UI (although I can't recall and am not sure where that is stored now).

[cloneofghosts]

Hi @catchdave You can always email @alexander0042 at api@pirateweather.net with the information needed to look into this issue.

[cloneofghosts]

Hi @catchdave are you still having issues with hitting the API limit? V1.5.5 introduced the option to change the scan interval after you configured the integration so you should be able to change it if you're still having issues.

[catchdave]

I was able to get it working. Alexander mentioned that the previous version (when it was configured in yaml) had HA reusing API calls which is probably the reason why I never hit limits previously.

[cloneofghosts]

Good to know you got this working. Will close this issue but if it pops up again feel free to reply or create a new issue.