I'm using the requests extension called requests-toolbelt which allows for mult-threaded requests. I believe the algorithm feeds multiple URLs to a requests.Session object and threads them up in a pool.
Can this be adapted to do rate limit/throttling for APIs that limit requests to something like 20 requests per second?
I'm using the requests extension called requests-toolbelt which allows for mult-threaded requests. I believe the algorithm feeds multiple URLs to a requests.Session object and threads them up in a pool.
Can this be adapted to do rate limit/throttling for APIs that limit requests to something like 20 requests per second?
https://toolbelt.readthedocs.io/en/latest/