HTTP 429, too many requests

[flekschas]

When looping through the top rated books I get a HTTP 429 error, i.e. I performed too many requests in a given time. I was able to work around that by specifying a timeout every tenth requests for a couple of seconds.

So basically I added

import time

and then

for page in range(1, numPages + 1):
    if page % 10 == 0:
        time.sleep(3)  # Sleep for 3 seconds every tenth request
    httpResp = tmdb_api.get('https://api.themoviedb.org/3/movie/top_rated', params={'page': page})  #(1)

But I am not sure if thats the best way to do it.

[sstults]

Did you happen to see if there was an associated body with that error, or maybe a Retry-After header? The RFC suggests we might be able to find out exactly what the max request rate is.

[flekschas]

Here is the request's content.

Your request count (41) is over the allowed limit of 40.

Their website says:

Are there limitations on the number of requests? We currently rate limit requests to 30 requests every 10 seconds.

So it seems like sleeping 10 seconds every 40 requests works fine. (So their API allows more requests than their FAQ indicates.)

[softwaredoug]

@flekschas the strategy for these examples have changed, we include some prechewed data from TMDB with an appendix on how to build the data yourself (which has the right rate limitting). Thanks for reporting