meraki-analytics / cassiopeia

An all-inclusive Python framework for the Riot Games League of Legends API. Cass focuses on making the data easy and fun to work with, while providing all the tools necessary to create a website or do data analysis.
MIT License
552 stars 135 forks source link

Best way to implement rate limiting? #211

Closed abrahimladha closed 6 years ago

abrahimladha commented 6 years ago

Testing out one of the examples:

`

champion_id_to_name_mapping = {champion.id: champion.name for champion in cass.get_champions(region=region)}
played_champions = Counter()
for match in match_history:
    champion_id = match.participants[summoner.name].champion.id
    champion_name = champion_id_to_name_mapping[champion_id]
    played_champions[champion_name] += 1

Printing the first few elements looked like this: Leona 855 Caitlyn 27 Vladimir 26 Braum 23 Nocturne 22

Doing this a for a few other users quickly got my api key blacklisted. Did this really make over a thousand api calls? Is there a way I can get many many match histories in a few calls? or is every match always going to be a full api call? Right now I added another counter to stop after 50 match histories but this seems like a futile solution, since I am still making 50 calls for one users request. I don't really want to do a massive database system of match histories like how op.gg or other sites might do for match histories.

jjmaldonis commented 6 years ago

So this code only makes a number of calls equal to the length of your match history divided by 100 (plus a few extra for static data from ddragon). I have 2600 games played, so your code made ~ 26 calls for me.

That's not enough to get you blacklisted, even if you do this for a few other users. Also, we're almost positive our rate limiter is good and won't ever get you blacklisted. We respect all the rate limits that Riot has, and I've personally used it to make tens of thousands of calls over multiple days, so I'm pretty confident that Cass can't get you blacklisted.

Cass may "pause" to wait for enough time to prevent you from getting blacklisted, which can sometimes take 2-5 minutes (and sometimes longer) depending on your limits. If you got blacklisted, there has to be another script that's using your API key -- that's the only thing I can think of.

abrahimladha commented 6 years ago

Thank you. That was exactly the answer I wanted to hear. Looking now riot says that if im blacklisted my code will be 429 instead of 403. I have some other issue. I appreciate your work!

jjmaldonis commented 6 years ago

403s mean your API key isn't valid. It probably expired, you have to renew it every 24 hours, so that's my guess.

abrahimladha commented 6 years ago

Yeah the issue was I was calling cass.set_riot_api_key, then i was doing config = cass.get_default_config, modifying config, then cass.apply_settings(config), in that order. I was overridding the me setting the riot api key.