search

stefan-jansen / zipline-reloaded

Zipline, a Pythonic Algorithmic Trading Library
https://zipline.ml4trading.io
Apache License 2.0
1.13k stars 210 forks source link

zipline ingest quandl download Issue #205

Closed Katya047 closed 1 year ago

Katya047 commented 1 year ago

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment(description of issue is under environment only):

Environment

* Operating System: MacOS * Python Version: 3.8.17 * Python Bitness: 64 * How did you install Zipline: mamba * Python packages: `$ pip freeze` or `$ conda list` ``` Name Version Build Channel alembic 1.11.1 pyhd8ed1ab_0 conda-forge anyio 3.7.1 pyhd8ed1ab_0 conda-forge appdirs 1.4.4 pyh9f0ad1d_0 conda-forge appnope 0.1.3 pyhd8ed1ab_0 conda-forge argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge argon2-cffi-bindings 21.2.0 py38hef030d1_3 conda-forge asttokens 2.2.1 pyhd8ed1ab_0 conda-forge async-lru 2.0.2 pyhd8ed1ab_0 conda-forge attrs 23.1.0 pyh71513ae_1 conda-forge babel 2.12.1 pyhd8ed1ab_1 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 pyhd8ed1ab_3 conda-forge backports.functools_lru_cache 1.6.5 pyhd8ed1ab_0 conda-forge bcolz-zipline 1.2.6 py38hbd87e4b_0 conda-forge beautifulsoup4 4.12.2 pyha770c72_0 conda-forge bleach 6.0.0 pyhd8ed1ab_0 conda-forge blosc 1.21.4 heccf04b_0 conda-forge bottleneck 1.3.7 py38hbd87e4b_0 conda-forge brotli 1.0.9 hb7f2c08_9 conda-forge brotli-bin 1.0.9 hb7f2c08_9 conda-forge brotli-python 1.0.9 py38h4cd09af_9 conda-forge bzip2 1.0.8 h0d85af4_4 conda-forge c-ares 1.19.1 h0dc2134_0 conda-forge c-blosc2 2.10.0 h0dcb076_0 conda-forge ca-certificates 2023.5.7 h8857fd0_0 conda-forge cached-property 1.5.2 hd8ed1ab_1 conda-forge cached_property 1.5.2 pyha770c72_1 conda-forge certifi 2023.5.7 pyhd8ed1ab_0 conda-forge cffi 1.15.1 py38hb368cf1_3 conda-forge charset-normalizer 3.2.0 pyhd8ed1ab_0 conda-forge click 8.1.4 unix_pyh707e725_0 conda-forge comm 0.1.3 pyhd8ed1ab_0 conda-forge contourpy 1.1.0 py38h15a1a5b_0 conda-forge cryptography 41.0.1 py38he13da33_0 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge debugpy 1.6.7 py38h4cd09af_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge empyrical-reloaded 0.5.9 pyhd8ed1ab_0 conda-forge entrypoints 0.4 pyhd8ed1ab_0 conda-forge exceptiongroup 1.1.2 pyhd8ed1ab_0 conda-forge exchange-calendars 4.2.6 pyhd8ed1ab_1 conda-forge executing 1.2.0 pyhd8ed1ab_0 conda-forge flit-core 3.9.0 pyhd8ed1ab_0 conda-forge fonttools 4.40.0 py38hcafd530_0 conda-forge freetype 2.12.1 h3f81eb7_1 conda-forge frozendict 2.3.8 py38hcafd530_0 conda-forge greenlet 2.0.2 py38h940360d_1 conda-forge h5py 3.9.0 nompi_py38h7a8cfb6_101 conda-forge hdf5 1.14.1 nompi_hedada53_100 conda-forge html5lib 1.1 pyh9f0ad1d_0 conda-forge icu 72.1 h7336db1_0 conda-forge idna 3.4 pyhd8ed1ab_0 conda-forge importlib-metadata 6.8.0 pyha770c72_0 conda-forge importlib-resources 6.0.0 pyhd8ed1ab_0 conda-forge importlib_metadata 6.8.0 hd8ed1ab_0 conda-forge importlib_resources 6.0.0 pyhd8ed1ab_0 conda-forge intervaltree 3.1.0 pyhd8ed1ab_1 conda-forge ipykernel 6.24.0 pyh5fb750a_0 conda-forge ipython 8.12.0 pyhd1c38e8_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge iso3166 2.1.1 pyhd8ed1ab_0 conda-forge iso4217 1.9.20220401 pyhd8ed1ab_0 conda-forge jedi 0.18.2 pyhd8ed1ab_0 conda-forge jinja2 3.1.2 pyhd8ed1ab_1 conda-forge json5 0.9.14 pyhd8ed1ab_0 conda-forge jsonschema 4.18.0 pyhd8ed1ab_0 conda-forge jsonschema-specifications 2023.6.1 pyhd8ed1ab_0 conda-forge jupyter-lsp 2.2.0 pyhd8ed1ab_0 conda-forge jupyter_client 8.3.0 pyhd8ed1ab_0 conda-forge jupyter_core 5.3.1 py38h50d1736_0 conda-forge jupyter_events 0.6.3 pyhd8ed1ab_0 conda-forge jupyter_server 2.7.0 pyhd8ed1ab_0 conda-forge jupyter_server_terminals 0.4.4 pyhd8ed1ab_1 conda-forge jupyterlab 4.0.2 pyhd8ed1ab_0 conda-forge jupyterlab_pygments 0.2.2 pyhd8ed1ab_0 conda-forge jupyterlab_server 2.23.0 pyhd8ed1ab_0 conda-forge kiwisolver 1.4.4 py38h98b9b1b_1 conda-forge korean_lunar_calendar 0.3.1 pyhd8ed1ab_0 conda-forge krb5 1.20.1 h049b76e_0 conda-forge lcms2 2.15 h2dcdeff_1 conda-forge lerc 4.0.0 hb486fe8_0 conda-forge libaec 1.0.6 hf0c8a7f_1 conda-forge libblas 3.9.0 17_osx64_openblas conda-forge libbrotlicommon 1.0.9 hb7f2c08_9 conda-forge libbrotlidec 1.0.9 hb7f2c08_9 conda-forge libbrotlienc 1.0.9 hb7f2c08_9 conda-forge libcblas 3.9.0 17_osx64_openblas conda-forge libcurl 8.1.2 hbee3ae8_0 conda-forge libcxx 16.0.6 hd57cbcb_0 conda-forge libdeflate 1.18 hac1461d_0 conda-forge libedit 3.1.20191231 h0678c8f_2 conda-forge libev 4.33 haf1e3a3_1 conda-forge libffi 3.4.2 h0d85af4_5 conda-forge libgfortran 5.0.0 11_3_0_h97931a8_31 conda-forge libgfortran5 12.2.0 he409387_31 conda-forge libiconv 1.17 hac89ed1_0 conda-forge libjpeg-turbo 2.1.5.1 hb7f2c08_0 conda-forge liblapack 3.9.0 17_osx64_openblas conda-forge libnghttp2 1.52.0 he2ab024_0 conda-forge libopenblas 0.3.23 openmp_h429af6e_0 conda-forge libpng 1.6.39 ha978bb4_0 conda-forge libsodium 1.0.18 hbcb3906_1 conda-forge libsqlite 3.42.0 h58db7d2_0 conda-forge libssh2 1.11.0 hd019ec5_0 conda-forge libta-lib 0.4.0 hb7f2c08_1 conda-forge libtiff 4.5.1 hf955e92_0 conda-forge libwebp-base 1.3.1 h0dc2134_0 conda-forge libxcb 1.15 hb7f2c08_0 conda-forge libxml2 2.11.4 hd95e348_0 conda-forge libxslt 1.1.37 h20bfa82_1 conda-forge libzlib 1.2.13 h8a1eda9_5 conda-forge llvm-openmp 16.0.6 hff08bdf_0 conda-forge logbook 1.5.3 py38hef030d1_7 conda-forge lru-dict 1.2.0 py38hcafd530_0 conda-forge lxml 4.9.3 py38h2d3ed51_0 conda-forge lz4-c 1.9.4 hf0c8a7f_0 conda-forge mako 1.2.4 pyhd8ed1ab_0 conda-forge markupsafe 2.1.3 py38hcafd530_0 conda-forge matplotlib 3.7.2 py38h50d1736_0 conda-forge matplotlib-base 3.7.2 py38h6e6aad2_0 conda-forge matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge mistune 3.0.0 pyhd8ed1ab_0 conda-forge multipledispatch 0.6.0 py_0 conda-forge multitasking 0.0.9 pyhd8ed1ab_0 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge nb_conda 2.2.1 unix_6 conda-forge nb_conda_kernels 2.3.1 py38h50d1736_2 conda-forge nbclassic 1.0.0 pyhb4ecaf3_1 conda-forge nbclient 0.8.0 pyhd8ed1ab_0 conda-forge nbconvert 7.6.0 pyhd8ed1ab_0 conda-forge nbconvert-core 7.6.0 pyhd8ed1ab_0 conda-forge nbconvert-pandoc 7.6.0 pyhd8ed1ab_0 conda-forge nbformat 5.9.1 pyhd8ed1ab_0 conda-forge ncurses 6.4 hf0c8a7f_0 conda-forge nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge networkx 3.1 pyhd8ed1ab_0 conda-forge notebook 6.5.4 pyha770c72_0 conda-forge notebook-shim 0.2.3 pyhd8ed1ab_0 conda-forge numexpr 2.8.4 py38hdaefe69_0 conda-forge numpy 1.24.4 py38h9a4a08f_0 conda-forge openjpeg 2.5.0 h13ac156_2 conda-forge openssl 3.1.1 h8a1eda9_1 conda-forge overrides 7.3.1 pyhd8ed1ab_0 conda-forge packaging 23.1 pyhd8ed1ab_0 conda-forge pandas 1.5.3 py38hec72209_1 conda-forge pandas-datareader 0.10.0 pyh6c4a22f_0 conda-forge pandoc 3.1.3 h9d075a6_0 conda-forge pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge parso 0.8.3 pyhd8ed1ab_0 conda-forge patsy 0.5.3 pyhd8ed1ab_0 conda-forge pexpect 4.8.0 pyh1a96a4e_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 10.0.0 py38h16710f9_0 conda-forge pip 23.1.2 pyhd8ed1ab_0 conda-forge pkgutil-resolve-name 1.3.10 pyhd8ed1ab_0 conda-forge platformdirs 3.8.1 pyhd8ed1ab_0 conda-forge pooch 1.7.0 pyha770c72_3 conda-forge prometheus_client 0.17.0 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.39 pyha770c72_0 conda-forge prompt_toolkit 3.0.39 hd8ed1ab_0 conda-forge psutil 5.9.5 py38hef030d1_0 conda-forge pthread-stubs 0.4 hc929b4f_1001 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge py-cpuinfo 9.0.0 pyhd8ed1ab_0 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pygments 2.15.1 pyhd8ed1ab_0 conda-forge pyluach 2.2.0 pyhd8ed1ab_0 conda-forge pyobjc-core 9.2 py38h095c2e5_0 conda-forge pyobjc-framework-cocoa 9.2 py38h095c2e5_0 conda-forge pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge pytables 3.8.0 py38h3c32be3_2 conda-forge python 3.8.17 hf9b03c3_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-fastjsonschema 2.17.1 pyhd8ed1ab_0 conda-forge python-interface 1.6.0 py_0 conda-forge python-json-logger 2.0.7 pyhd8ed1ab_0 conda-forge python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge python_abi 3.8 3_cp38 conda-forge pytz 2023.3 pyhd8ed1ab_0 conda-forge pyyaml 6.0 py38hef030d1_5 conda-forge pyzmq 25.1.0 py38h3b70857_0 conda-forge readline 8.2 h9e318b2_1 conda-forge referencing 0.29.1 pyhd8ed1ab_0 conda-forge requests 2.31.0 pyhd8ed1ab_0 conda-forge rfc3339-validator 0.1.4 pyhd8ed1ab_0 conda-forge rfc3986-validator 0.1.1 pyh9f0ad1d_0 conda-forge rpds-py 0.8.8 py38h7510fb3_0 conda-forge scipy 1.10.1 py38h9cf86d3_3 conda-forge send2trash 1.8.2 pyhd1c38e8_0 conda-forge setuptools 68.0.0 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.10 h225ccf5_0 conda-forge sniffio 1.3.0 pyhd8ed1ab_0 conda-forge sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge soupsieve 2.3.2.post1 pyhd8ed1ab_0 conda-forge sqlalchemy 1.4.49 py38hcafd530_0 conda-forge stack_data 0.6.2 pyhd8ed1ab_0 conda-forge statsmodels 0.14.0 py38hd2faf92_1 conda-forge ta-lib 0.4.19 py38hd2faf92_4 conda-forge terminado 0.17.1 pyhd1c38e8_0 conda-forge tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge tk 8.6.12 h5dbffcc_0 conda-forge tomli 2.0.1 pyhd8ed1ab_0 conda-forge toolz 0.12.0 pyhd8ed1ab_0 conda-forge tornado 6.3.2 py38hcafd530_0 conda-forge trading-calendars 2.1.1 pyhd3deb0d_0 conda-forge traitlets 5.9.0 pyhd8ed1ab_0 conda-forge typing-extensions 4.7.1 hd8ed1ab_0 conda-forge typing_extensions 4.7.1 pyha770c72_0 conda-forge typing_utils 0.1.0 pyhd8ed1ab_0 conda-forge unicodedata2 15.0.0 py38hef030d1_0 conda-forge urllib3 2.0.3 pyhd8ed1ab_1 conda-forge wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge webencodings 0.5.1 py_1 conda-forge websocket-client 1.6.1 pyhd8ed1ab_0 conda-forge wheel 0.40.0 pyhd8ed1ab_0 conda-forge xorg-libxau 1.0.11 h0dc2134_0 conda-forge xorg-libxdmcp 1.1.3 h35c211d_0 conda-forge xz 5.2.6 h775f41a_0 conda-forge yaml 0.2.5 h0d85af4_2 conda-forge yfinance 0.2.22 pyhd8ed1ab_0 conda-forge zeromq 4.3.4 he49afe7_1 conda-forge zipline-reloaded 2.4 py38hbd87e4b_1 conda-forge zipp 3.16.0 pyhd8ed1ab_0 conda-forge zlib-ng 2.0.7 hb7f2c08_0 conda-forge zstd 1.5.2 h829000d_7 conda-forge ``` Now that you know a little about me, let me tell you about the issue I am having: # Description of Issue * What did you expect to happen? The data should be downloaded from Quandl * What happened instead? I am getting the below error after entering ``` zipline ingest -b quandl [2023-07-10 04:13:45.782807] INFO: zipline.data.bundles.core: Ingesting quandl. [2023-07-10 04:13:45.782908] INFO: zipline.data.bundles.quandl: Downloading WIKI metadata. [2023-07-10 04:15:01.876197] ERROR: zipline.data.bundles.quandl: Exception raised reading Quandl data. Retrying. Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1354, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 951, in send self.connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1418, in connect super().connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 922, in connect self.sock = self._create_connection( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 808, in create_connection raise err File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 796, in create_connection sock.connect(sa) TimeoutError: [Errno 60] Operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/zipline/data/bundles/quandl.py", line 75, in fetch_data_table metadata = pd.read_csv(format_metadata_url(api_key)) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv return _read(filepath_or_buffer, kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 577, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__ self._engine = self._make_engine(f, self.engine) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1661, in _make_engine self.handles = get_handle( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 716, in get_handle ioargs = _get_filepath_or_buffer( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 368, in _get_filepath_or_buffer with urlopen(req_info) as req: File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 270, in urlopen return urllib.request.urlopen(*args, **kwargs) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 525, in open response = self._open(req, data) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 542, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 502, in _call_chain result = func(*args) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1397, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1357, in do_open raise URLError(err) urllib.error.URLError: [2023-07-10 04:15:01.889428] INFO: zipline.data.bundles.quandl: Downloading WIKI metadata. [2023-07-10 04:16:17.283948] ERROR: zipline.data.bundles.quandl: Exception raised reading Quandl data. Retrying. Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1354, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 951, in send self.connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1418, in connect super().connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 922, in connect self.sock = self._create_connection( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 808, in create_connection raise err File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 796, in create_connection sock.connect(sa) TimeoutError: [Errno 60] Operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/zipline/data/bundles/quandl.py", line 75, in fetch_data_table metadata = pd.read_csv(format_metadata_url(api_key)) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv return _read(filepath_or_buffer, kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 577, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__ self._engine = self._make_engine(f, self.engine) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1661, in _make_engine self.handles = get_handle( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 716, in get_handle ioargs = _get_filepath_or_buffer( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 368, in _get_filepath_or_buffer with urlopen(req_info) as req: File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 270, in urlopen return urllib.request.urlopen(*args, **kwargs) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 525, in open response = self._open(req, data) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 542, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 502, in _call_chain result = func(*args) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1397, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1357, in do_open raise URLError(err) urllib.error.URLError: [2023-07-10 04:16:17.288326] INFO: zipline.data.bundles.quandl: Downloading WIKI metadata. [2023-07-10 04:17:32.408049] ERROR: zipline.data.bundles.quandl: Exception raised reading Quandl data. Retrying. Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1354, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 951, in send self.connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1418, in connect super().connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 922, in connect self.sock = self._create_connection( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 808, in create_connection raise err File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 796, in create_connection sock.connect(sa) TimeoutError: [Errno 60] Operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/zipline/data/bundles/quandl.py", line 75, in fetch_data_table metadata = pd.read_csv(format_metadata_url(api_key)) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv return _read(filepath_or_buffer, kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 577, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__ self._engine = self._make_engine(f, self.engine) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1661, in _make_engine self.handles = get_handle( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 716, in get_handle ioargs = _get_filepath_or_buffer( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 368, in _get_filepath_or_buffer with urlopen(req_info) as req: File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 270, in urlopen return urllib.request.urlopen(*args, **kwargs) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 525, in open response = self._open(req, data) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 542, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 502, in _call_chain result = func(*args) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1397, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1357, in do_open raise URLError(err) urllib.error.URLError: [2023-07-10 04:17:32.412007] INFO: zipline.data.bundles.quandl: Downloading WIKI metadata. [2023-07-10 04:18:47.571334] ERROR: zipline.data.bundles.quandl: Exception raised reading Quandl data. Retrying. Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1354, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 951, in send self.connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1418, in connect super().connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 922, in connect self.sock = self._create_connection( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 808, in create_connection raise err File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 796, in create_connection sock.connect(sa) TimeoutError: [Errno 60] Operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/zipline/data/bundles/quandl.py", line 75, in fetch_data_table metadata = pd.read_csv(format_metadata_url(api_key)) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv return _read(filepath_or_buffer, kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 577, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__ self._engine = self._make_engine(f, self.engine) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1661, in _make_engine self.handles = get_handle( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 716, in get_handle ioargs = _get_filepath_or_buffer( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 368, in _get_filepath_or_buffer with urlopen(req_info) as req: File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 270, in urlopen return urllib.request.urlopen(*args, **kwargs) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 525, in open response = self._open(req, data) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 542, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 502, in _call_chain result = func(*args) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1397, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1357, in do_open raise URLError(err) urllib.error.URLError: [2023-07-10 04:18:47.575059] INFO: zipline.data.bundles.quandl: Downloading WIKI metadata. [2023-07-10 04:20:02.702916] ERROR: zipline.data.bundles.quandl: Exception raised reading Quandl data. Retrying. Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1354, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 951, in send self.connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 1418, in connect super().connect() File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/http/client.py", line 922, in connect self.sock = self._create_connection( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 808, in create_connection raise err File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/socket.py", line 796, in create_connection sock.connect(sa) TimeoutError: [Errno 60] Operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/zipline/data/bundles/quandl.py", line 75, in fetch_data_table metadata = pd.read_csv(format_metadata_url(api_key)) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv return _read(filepath_or_buffer, kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 577, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__ self._engine = self._make_engine(f, self.engine) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1661, in _make_engine self.handles = get_handle( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 716, in get_handle ioargs = _get_filepath_or_buffer( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 368, in _get_filepath_or_buffer with urlopen(req_info) as req: File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/pandas/io/common.py", line 270, in urlopen return urllib.request.urlopen(*args, **kwargs) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 525, in open response = self._open(req, data) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 542, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 502, in _call_chain result = func(*args) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1397, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/urllib/request.py", line 1357, in do_open raise URLError(err) urllib.error.URLError: Traceback (most recent call last): File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/bin/zipline", line 11, in sys.exit(main()) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/zipline/__main__.py", line 389, in ingest bundles_module.ingest( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/zipline/data/bundles/core.py", line 445, in ingest bundle.ingest( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/zipline/data/bundles/quandl.py", line 184, in quandl_bundle raw_data = fetch_data_table( File "/Users/avneeshyadav/Desktop/Python/anaconda3/envs/zipline/lib/python3.8/site-packages/zipline/data/bundles/quandl.py", line 97, in fetch_data_table raise ValueError( ValueError: Failed to download Quandl data after 5 attempts. ``` Here is how you can reproduce this issue on your machine: ## Reproduction Steps 1. 2. 3. ... ## What steps have you taken to resolve this already? Double-checked my API key and also did `conda env config vars list` to check the value of QUANDL_API_KEY # Anything else? Sincerely, Avneesh
MBounouar commented 1 year ago

It looks unlikely that the problem comes from zipline. Either the site was down (it currently works) or it's an issue with your network (maybe you firewall?)

Katya047 commented 1 year ago

@MBounouar Firewall is already turned off and I am trying this for the last 3 days but getting the same output. Any other points that I can check?

MBounouar commented 1 year ago

The issue is with you socket connection. You need to first check that you can indeed create a connection, see for instance socket doc

Katya047 commented 1 year ago

I was unable to ingest the data with the below command: zipline ingest I downloaded the data manually in zip format and passed the path of my zip file on Line no. 88 in quandl.py I also commented out lines 75 to 85. Now, the same command worked and it processed the data from the zip file from my system instead of downloading it from the internet.