quandl / quandl-python

MIT License
1.39k stars 340 forks source link

cache - reduce number of API calls - provide offline access of ever called data #30

Closed femtotrader closed 8 years ago

femtotrader commented 9 years ago

Hello,

It will be nice to provide a cache mechanism to Quandl Python. requests and requests-cache could help. You can see my code here https://github.com/femtotrader/pandas_datareaders/

Kind regards

jbyck commented 8 years ago

Thanks femotrader, but we do not have any plans to introduce a caching layer into the package. We view this package as thin layer around our API, and we are hesitant to add additional complexity, which may be confusing to some users.

femtotrader commented 8 years ago

Is it still something you don't want ?

see http://pandas-datareader.readthedocs.io/en/latest/cache.html

I had a look at https://github.com/quandl/quandl-python/blob/e2540dc8333c03802583344820bb0dd6b93f07df/quandl/connection.py#L38

I think you just need an extra parameter (session for example) session will be set to None by default

You will have

https://github.com/quandl/quandl-python/blob/e2540dc8333c03802583344820bb0dd6b93f07df/quandl/connection.py#L41

    @classmethod
    def execute_request(cls, http_verb, url, **options, session=None):
        try:
            if session is None:
                session = requests.Session()
            func = getattr(session, http_verb)
            response = func(url, **options)

I don't think that's a very invasive modification but can be very useful!

femtotrader commented 8 years ago

in fact... to be even less invasive session could come from options kwargs.

@classmethod
    def execute_request(cls, http_verb, url, **options):
        try:
            if 'session' not in options:
                session = requests.Session()
            else:
                session = options['session']
            func = getattr(session, http_verb)
            response = func(url, **options)
ericremoreynolds commented 8 years ago

One workaround is to use requests_cache.install_cache to monkey patch all HTTP requests, including the ones Quandl produces.

femtotrader commented 8 years ago

Thanks @ericremoreynolds but passing session would be a much cleaner approach than monkey patch !