API Limit Hit

[mermelmadness]

Woke up this morning to see the following message in Attributes:

  Cached data: API_LIMIT hit.  No competition found for 'MIA' between
  2023-03-08T00:00Z and 2023-03-14T00:00Z

I don't remember reading anywhere in the Wiki about an API limit. MIA definitely has games coming up, tomorrow in fact.

[NRutt2005]

That isn't good. So it be that it is trying to grab the game data from the API too much. Maybe there is a way to limit the amount of times it grabs data when there isn't a live game? Not sure if that is even possible.

[mermelmadness]

Maybe this has to do with ESPN removing their Dev Center page. Maybe they're reworking their APIs to be a premium subscription service.

To be fair, I do have 7 instances of Team Tracker integrated into my HA (and had an 8th for a couple of days for testing), but I get data for the rest of them just fine, and am continuing to get live data.

Please see what could be the issue. If it involves reducing the amount of calls made (I know PirateWeather limits free users to 25,000 per month) that would suck. I'd be happy to donate to keep the API data coming...

[vasqued2]

The API limits the number of games returned to 25 by default. It returns games from yesterday and then the next 6 days until it hits the 25 game limit. There have been several times during the pre-season baseball that there have been over 25 games in a 2 day period and the late games on the second day (today) gets dropped because of the limit. I suspect that's what's happening here.

Once the season starts, if it keeps happening, I will have to have it start w/ today's games instead of yesterdays. This will mean post-game results will drop off faster but upcoming games will be returned.

Haven't seen other sports hit the 25 limit. Just baseball.

[mermelmadness]

MLB is unique in that each team typically has 5-6 games (sometimes 7) in that 7-day window, and if I have 5 MLB teams I'm following, that's 35. What would it take to up the amount of games returned beyond 25? If the POST falls off a bit faster, since most games are night games, that wouldn't be the worst thing in the world. Any way to change the number of look-ahead days to, maybe, 4 or 5 instead of 6?

It may be unique per sport because I also have NCAAB (both Top 25 and id: 50) integrated as well, though that will drop off after April 3rd.

[mermelmadness]

Based on what you said, though, it looks like that if I do run into the limit, it'll reset the next day.

[vasqued2]

The API returns all games in the league for the requested date range until it hits 25 games returned. The sensor then searches those 25 games to find your team. You can't specify you only want games for MIA. It returns them all. Since MIA isn't one of the first 25 teams it doesn't get data returned for it. It gives that error message so you know MIA wasn't in the first 25 returned.

[mermelmadness]

That's insane, though. 4 days a week there are 15 games per day. The other 3 days there are at least 7. That means in a given 7-day span there is, at minimum, 81 games to be returned. Even in leagues that don't play every day, like NHL, there are about 50-60 games in a 7-day span. How does any application use the ESPN API with such limitations?

[vasqued2]

While games are going on, the API gets called every 5 seconds. That's a ton of games and info being returned every minute. To limit the amount of data returned, they cap it to 25 games. It's a free, undocumented API. ESPN can put whatever limits they want on it to control usage and we have to be grateful that it exists and use it accordingly.

The alternative is what people describe about. ESPN will throttle the number of calls or move it to a premium service.

And you don't need all 50-60 NHL games returned at once. The sensor only displays 1 game per team so if you go back one day and then look forward, it will find the game on or before gameday. Only baseball w/ tons of back-to-backs and double headers hits the limit.

[NRutt2005]

@mermelmadness when you say you have NCAAB, I'm assuming that is NCAA Basketball and not Baseball? I really would like to get NCAA Baseball and Softball showing up but I don't know how...

[mermelmadness]

@NRutt2005 Yes, NCAA Basketball

[mermelmadness]

@vasqued2 I see. Hmm. So, did I just get unlucky that one of the teams I follow was beyond the first 25 games for the day, or is it because I have 7 integrations, 2 of which span the entire league?

[vasqued2]

You were just unlucky. It seems to generally sort them based on start time so late/west coast games get cut off most often but not always.

If you have multiple sensors from the same league it will cache the API data and use it both both teams instead of recalling the API again so that saves some API calls.

[NRutt2005]

@vasqued2 any chance of getting College Baseball and College Softball added to the integration? I found the baseball and softball api url but I can't seem to get the data to come in.

Thanks!

[mermelmadness]

I assume that's only if the teams I integrated are playing each other. Damn. I integrated all 5 teams from the NL East so I could create live standings next to my live scoreboard.

With MIA at null, it ruins the whole formula.

nl_east:
        friendly_name: "NL East"
        value_template: >-
          {% set teams = [
            {'name': 'NYM', 'record': state_attr('sensor.mets','team_record'), 'percentage': '{:.3f}'.format(float(states('sensor.mets_win_percentage'))) | replace('0.','.') },
            {'name': 'PHI', 'record': state_attr('sensor.phillies','team_record'), 'percentage': '{:.3f}'.format(float(states('sensor.phillies_win_percentage'))) | replace('0.','.') },
            {'name': 'MIA', 'record': state_attr('sensor.marlins','team_record'), 'percentage': '{:.3f}'.format(float(states('sensor.marlins_win_percentage'))) | replace('0.','.') },
            {'name': 'ATL', 'record': state_attr('sensor.braves','team_record'), 'percentage': '{:.3f}'.format(float(states('sensor.braves_win_percentage'))) | replace('0.','.') },
            {'name': 'WSH', 'record': state_attr('sensor.nationals','team_record'), 'percentage': '{:.3f}'.format(float(states('sensor.nationals_win_percentage'))) | replace('0.','.') },
          ] %}
          {% set sorted_teams = teams | sort(attribute='percentage', reverse=true) %}
          {% set formatted_teams = sorted_teams | map(attribute='name') | join(', ') %}
          {% set formatted_percentages = sorted_teams | map(attribute='percentage') | join(', ') %}
          {% set formatted_records = sorted_teams | map(attribute='record') | join(', ') %}
          Teams: {{ formatted_teams }} 
          Team Records: {{ formatted_records }}
          Winning Percentages: {{ formatted_percentages }}

Any suggestions on how I could possibly avoid this going forward, if at all? Does ESPN have a paid sub that will give you more than 25 games?

[NRutt2005]

I don't think they have a premium API subscription. If they did, I bet it wouldn't be cheap. It's ESPN after all. The API is not even published anywhere that I've been able to find. I wish they would at least have a API info site that provides some kind of break down of limitations and all.

[mermelmadness]

With midnight came a refresh and MIA came back. Guess it'll just be like this throughout the season....or maybe just Spring Training. Who knows?

@vasqued2 If you have any ideas on how to avoid this for MLB I'd be open to any suggestions. Let me know if I can help in any way if you plan to address this. If you don't, thanks for making this integration. I love it.

[mermelmadness]

This time, today's game isn't showing up. The team sensor is still in POST

Is this related to the 25 game limit?

The game is definitely listed in the API

{"id":"401480916","uid":"s:1~l:10~e:401480916","date":"2023-03-20T22:05Z","name":"New York Mets at Washington Nationals","shortName":"NYM @ WSH","season":{"year":2023,"type":1,"slug":"preseason"},...

[vasqued2]

You can try the pre-release version in master to see if it resolves it or wait a couple days for the official release.

[mermelmadness]

I'll wait. Not in a rush. Thanks for looking into this. Will report after update becomes available in HACS.

[vasqued2]

Sounds good. Trying to release before opening day because I know it'll hit the limit then, but having some issues w/ HACS right now.

Basically, it will only return games for a single day and rely on the ESPN API to determine when to switch to the current day games. Near as I can tell, it returns yesterday's games until about noon EST and then cuts over to the current day. Downside it if there is not game for the day the API returns data for, it will return a status of NOT_FOUND.

[mermelmadness]

For baseball I think that's fine. Other sports that don't play every day I can see that being annoying, but 95% of the time the status will either be PRE or POST

I know I'm one person, but I'm fine with this.

In my observations, the state typically changes from POST to PRE just after midnight if the game is over, but in spring training most games are 1pm starts, so that may change with more night games.

[mermelmadness]

Also, I don't know enough to download the master code and install it manually. I'm sure I could figure it out, but I'm tired.

[vasqued2]

You should be able to upgrade to v0.7.2 and see how it goes. Games should cutover from yesterday to today around noon EST. I haven't paid attention to the exact time.

[mermelmadness]

Updated to 0.7.2 and it's showing today's game as PRE. This is the last preseason game and opening day is Thursday, so there'll be a good sized gap between games. I'll report on what I see.

[vasqued2]

Thanks. I'll leave this open and you can let me know if there are any issues.

[mermelmadness]

I can report that all 5 MLB teams I have integrated (the whole NL East) are all showing PRE as their state.

[vasqued2]

Made it through opening day with all teams playing and API call is working as intended, pulling back a single day's games (i.e. either yesterday or today depending on the cutover time).

Closing as resolved, although opening another baseball related defect related to the NOT_FOUND date.