When working in a notebook or when fetching from static endpoints, there often come times where the same endpoint is called, which would benefit from having a cached URL session. Also sometimes when I need to make a large volume of requests in a short amount of time, it risks hitting the AV rate limit (which I believe is 25/day for free tier, 75/min for the next tier, not sure after that). It'd be nice if I can pass in a custom requests session that handles these things. I know that for yfinance, you can construct a custom session and use it with the Ticker class like this
from requests import Session
from requests_cache import CacheMixin, SQLiteCache
from requests_ratelimiter import LimiterMixin, MemoryQueueBucket
from pyrate_limiter import Duration, RequestRate, Limiter
class CachedLimiterSession(CacheMixin, LimiterMixin, Session):
pass
session = CachedLimiterSession(
limiter=Limiter(RequestRate(2, Duration.SECOND*5)), # max 2 requests per 5 seconds
bucket_class=MemoryQueueBucket,
backend=SQLiteCache("yfinance.cache"),
)
ticker = yf.Ticker('msft', session=session)
# The scraped response will be stored in the cache
ticker.actions
I think the AV client could also benefit from something like this. Maybe add an optional attribute session to class AlphaVantage(object): so that def _handle_api_call(self, url): can default to use the custom session if available?
When working in a notebook or when fetching from static endpoints, there often come times where the same endpoint is called, which would benefit from having a cached URL session. Also sometimes when I need to make a large volume of requests in a short amount of time, it risks hitting the AV rate limit (which I believe is 25/day for free tier, 75/min for the next tier, not sure after that). It'd be nice if I can pass in a custom requests session that handles these things. I know that for
yfinance
, you can construct a custom session and use it with theTicker
class like thisI think the AV client could also benefit from something like this. Maybe add an optional attribute
session
toclass AlphaVantage(object):
so thatdef _handle_api_call(self, url):
can default to use the custom session if available?