quandl / quandl-python

MIT License
1.37k stars 336 forks source link

Datatable filtering bug #96

Closed casperkaae closed 6 years ago

casperkaae commented 6 years ago

Hi.

I think there might be a bug in the quandl.get_table when trying to filter the data. Specifically i try to filter the 'ZACKS/EEH' datase using the "obs_date" column to only download recent data. If i try to download the last 20 days the code below works fine, filters correctly, and i can see data up until yesterday. However trying to download for the e.g the last 10 days the code fails with the error:

QuandlError: (Status 500) Something went wrong. Please try again. If you continue to have problems, please contact us at connect@quandl.com.

Code example

database = {
    'dataset': 'ZACKS/EEH',
    'date_filter_column': 'obs_date'
}

import quandl
import datetime

n_days_back = 10 #change to e.g. 20 to make the code work

today = datetime.date.today()
start_delta = datetime.timedelta(days=n_days_back)
start_of_week = today - start_delta
start_date = "%d-%0.2d-%0.2d"%(start_of_week.year, start_of_week.month, start_of_week.day)
print(start_date)

options = {'datatable_code': database['dataset'],
           'paginate': True,
           database['date_filter_column']: {'gte': start_date}}
data = quandl.get_table(**options)

Traceback

QuandlError                               Traceback (most recent call last)
<ipython-input-149-cc398ea52228> in <module>()
     20            'paginate': True,
     21            database['date_filter_column']: {'gte': start_date}}
---> 22 data = quandl.get_table(**options)

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/get_table.py in get_table(datatable_code, **options)
     17     while True:
     18         next_options = copy.deepcopy(options)
---> 19         next_data = Datatable(datatable_code).data(params=next_options)
     20 
     21         if data is None:

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/model/datatable.py in data(self, **options)
     14     def data(self, **options):
     15         updated_options = Util.convert_options(**options)
---> 16         return Data.page(self, **updated_options)

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/operations/list.py in page(cls, datatable, **options)
     22         params = {'id': str(datatable.code)}
     23         path = Util.constructed_path(datatable.default_path(), params)
---> 24         r = Connection.request('get', path, **options)
     25         response_data = r.json()
     26         Util.convert_to_dates(response_data)

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/connection.py in request(cls, http_verb, url, **options)
     34         abs_url = '%s/%s' % (ApiConfig.api_base, url)
     35 
---> 36         return cls.execute_request(http_verb, abs_url, **options)
     37 
     38     @classmethod

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/connection.py in execute_request(cls, http_verb, url, **options)
     42             response = func(url, **options)
     43             if response.status_code < 200 or response.status_code >= 300:
---> 44                 cls.handle_api_error(response)
     45             else:
     46                 return response

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/connection.py in handle_api_error(cls, resp)
     64         # throw generic error
     65         if 'quandl_error' not in error_body:
---> 66             raise QuandlError(http_status=resp.status_code, http_body=resp.text)
     67 
     68         code = error_body['quandl_error']['code']

QuandlError: (Status 500) Something went wrong. Please try again. If you continue to have problems, please contact us at connect@quandl.com
junos commented 6 years ago

Hi @casperkaae,

I’ve attempted to reproduce the issue that you encountered, however I was able to retrieve data successfully.

If you are still encountering this issue, could you report back on the versions you are using for each please? For reference, I’m using Python 3.6.3 and the Quandl 3.2.1 package.