RomelTorres / alpha_vantage

A python wrapper for Alpha Vantage API for financial data.
MIT License
4.3k stars 741 forks source link

Add caching and rate limiting support? #376

Closed SnowCheetos closed 4 months ago

SnowCheetos commented 4 months ago

When working in a notebook or when fetching from static endpoints, there often come times where the same endpoint is called, which would benefit from having a cached URL session. Also sometimes when I need to make a large volume of requests in a short amount of time, it risks hitting the AV rate limit (which I believe is 25/day for free tier, 75/min for the next tier, not sure after that). It'd be nice if I can pass in a custom requests session that handles these things. I know that for yfinance, you can construct a custom session and use it with the Ticker class like this

from requests import Session
from requests_cache import CacheMixin, SQLiteCache
from requests_ratelimiter import LimiterMixin, MemoryQueueBucket
from pyrate_limiter import Duration, RequestRate, Limiter
class CachedLimiterSession(CacheMixin, LimiterMixin, Session):
    pass

session = CachedLimiterSession(
    limiter=Limiter(RequestRate(2, Duration.SECOND*5)),  # max 2 requests per 5 seconds
    bucket_class=MemoryQueueBucket,
    backend=SQLiteCache("yfinance.cache"),
)

ticker = yf.Ticker('msft', session=session)
# The scraped response will be stored in the cache
ticker.actions

I think the AV client could also benefit from something like this. Maybe add an optional attribute session to class AlphaVantage(object): so that def _handle_api_call(self, url): can default to use the custom session if available?

SnowCheetos commented 4 months ago

Created a pull request #377 for the implementation

AlphaVantageSupport commented 4 months ago

Thanks! We will keep the PR open and merge it upon community review.