celiao / tmdbsimple

A wrapper for The Movie Database API v3.
GNU General Public License v3.0
579 stars 122 forks source link

No rate limit management #55

Closed submarcos closed 4 years ago

submarcos commented 6 years ago

Is there a way to manage Rate limit properly ?

https://developers.themoviedb.org/3/getting-started/request-rate-limiting

I need to fix https://github.com/celiao/tmdbsimple/blob/master/tmdbsimple/base.py#L23 to extends request headers and propose a Pull request, is it ok for you ?

3lixy commented 6 years ago

I use this when i need rate limiting. It seems to work quite nicely:

https://gist.github.com/gregburek/1441055#gistcomment-2369461

celiao commented 5 years ago

Hey @3lixy, would you mind sharing some code snippets showing usage? Thanks!

3lixy commented 5 years ago

I currently use https://gist.github.com/gregburek/1441055 comment by oPromessa on Mar4

An example use (not tested):

import tmdbsimple as tmdb
tmdb.API_KEY = 'YOUR_API_KEY_HERE'
# allow access to rate_limited function from the above link (e.g. put it in the same python file you will use it or import it)

# Only allow this function be run once a second
@rate_limited(1)
def rate_limit_call(tmdb_object, function, *args, **kwargs):
  return getattr(tmdb_object, function)(*args, **kwargs)

if __name__ == "__main__":
  movie_id_list = [100,101,102,103]

  for movie_id in movie_list:
    # I don't think the movies __init__ actually does a http call so no rate limit needed
    movie = tmdb.Movies(movie_id)

    # The rate limit of movie.info()
    response = rate_limit_call(movie, 'info')

    # The rate limit of movie.releases()
    response = rate_limit_call(movie, 'releases')
3lixy commented 5 years ago

Is there a way to manage Rate limit properly ?

https://developers.themoviedb.org/3/getting-started/request-rate-limiting python I need to fix https://github.com/celiao/tmdbsimple/blob/master/tmdbsimple/base.py#L23 to extends request headers and propose a Pull request, is it ok for you ?

If you want to change the request headers you should be able to do something like (should only need to be done once each run time assuming it does not need to be changed during runtime):

import tmdbsimple as tmdb
tmdb.base.TMDB.headers['HEADER'] = 'HeaderValue'

or

import tmdbsimple as tmdb
tmdb.base.TMDB.headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'Connection': 'close'}
hackerjack42 commented 5 years ago

I was tinkering around with this in SickRage/SickBeard/Medusa and I found a very simple solution.

If the response code is 429, then sleep for the number of seconds specified in the "Retry-After" header value. This way your normal requests will never be throttled and it lets the TMDB API do all the work of telling you how long you have to wait for your next request.

diff --git a/lib/tmdbsimple/base.py b/lib/tmdbsimple/base.py
index d178f66b3..8c96ccfe1 100644
--- a/lib/tmdbsimple/base.py
+++ b/lib/tmdbsimple/base.py
@@ -13,6 +13,7 @@ Created by Celia Oakley on 2013-10-31.

 import json
 import requests
+import time

 class APIKeyError(Exception):
@@ -72,10 +73,16 @@ class TMDB(object):
         url = self._get_complete_url(path)
         params = self._get_params(params)

-        response = requests.request(
-            method, url, params=params,
-            data=json.dumps(payload) if payload else payload,
-            headers=self.headers)
+        retry = True
+        while retry:
+            retry = False
+            response = requests.request(
+                method, url, params=params,
+                data=json.dumps(payload) if payload else payload,
+                headers=self.headers)
+            if response.status_code == 429:
+                retry = True
+                time.sleep(int(response.headers['Retry-After']))

         response.raise_for_status()
         response.encoding = 'utf-8'
celiao commented 4 years ago

Great news! Rate limiting of The Movie Database API has been disabled as of December 16, 2019.

image

Closing this issue.