alphaville76 / sharadar_db_bundle

10 stars 8 forks source link

date error when installing from scratch #27

Closed vaa1234 closed 3 years ago

vaa1234 commented 3 years ago

I also found this error. When installing from scratch, NaT is substituted to date arg in quandl.get_table, which leads to an error. Should there really be a constant OLDEST_DATE_SEP?

[2021-04-13 15:52:36] INFO: Start ingesting SEP, SFP and SF1 data into /root/.zipline/data/sharadar/latest ...
[2021-04-13 15:52:36] INFO: Start loading sharadar metadata...
[2021-04-13 15:56:07] INFO: Adding SIDs to all stocks...
[2021-04-13 18:02:50] INFO: Adjusting for stock splits...
[2021-04-13 18:03:04] INFO: Start writing price data for 24497 equities.
[2021-04-14 17:08:32] INFO: Writing pricing data to '/root/.zipline/data/sharadar/latest/prices.sqlite'...
[2021-04-14 17:57:09] INFO: Creating dividends data...
[2021-04-14 17:59:05] INFO: Creating splits data...
[2021-04-14 17:59:09] INFO: Start writing 10030 splits and 378200 dividends data...
[2021-04-14 17:59:09] ERROR: no such column: inf: INSERT OR REPLACE INTO splits VALUES ('2504', 1385078400.0, inf, 122587.0)
[2021-04-14 17:59:28] DEBUG: Loading raw arrays for 10155 assets (<class 'numpy.ndarray'>).
[2021-04-14 18:00:47] INFO: Start writing equities and supplementary_mappings data...
[2021-04-14 18:01:19] INFO: Start creating company info dataframe...
[2021-04-14 18:01:35] INFO: Start creating Fundamentals dataframe from NaT...
[2021-04-14 18:01:49] INFO: Start creating daily metrics dataframe from NaT...
[2021-04-14 18:01:49] ERROR: Traceback (most recent call last):
  File "/root/anaconda3/envs/zipline/lib/python3.6/site-packages/sharadar_db_bundle-1.0-py3.6.egg/sharadar/loaders/ingest_sharadar.py", line 335, in ingest
    _ingest(start_date, calendar.last_session, calendar)
  File "/root/anaconda3/envs/zipline/lib/python3.6/site-packages/sharadar_db_bundle-1.0-py3.6.egg/sharadar/loaders/ingest_sharadar.py", line 245, in _ingest
    daily_df = fetch_table_by_date(env["QUANDL_API_KEY"], 'SHARADAR/DAILY', start_date_metrics, end_fetch_date)
  File "/root/anaconda3/envs/zipline/lib/python3.6/site-packages/sharadar_db_bundle-1.0-py3.6.egg/sharadar/util/quandl_util.py", line 109, in fetch_table_by_date
    df = quandl.get_table(table_name, date={'gte':start,'lte':end}, paginate=True)
  File "/root/anaconda3/envs/zipline/lib/python3.6/site-packages/quandl/get_table.py", line 19, in get_table
    next_data = Datatable(datatable_code).data(params=next_options)
  File "/root/anaconda3/envs/zipline/lib/python3.6/site-packages/quandl/model/datatable.py", line 31, in data
    return Data.page(self, **options)
  File "/root/anaconda3/envs/zipline/lib/python3.6/site-packages/quandl/operations/list.py", line 30, in page
    r = Connection.request(request_type, path, **updated_options)
  File "/root/anaconda3/envs/zipline/lib/python3.6/site-packages/quandl/connection.py", line 38, in request
    return cls.execute_request(http_verb, abs_url, **options)
  File "/root/anaconda3/envs/zipline/lib/python3.6/site-packages/quandl/connection.py", line 50, in execute_request
    cls.handle_api_error(response)
  File "/root/anaconda3/envs/zipline/lib/python3.6/site-packages/quandl/connection.py", line 114, in handle_api_error
    raise klass(message, resp.status_code, resp.text, resp.headers, code)
quandl.errors.quandl_error.InvalidRequestError: (Status 422) (Quandl Error QESx04) Type mismatch: One or more filters with type: 'date' and input: 'NaT' has a type mismatch
alphaville76 commented 3 years ago

fixed