Fresh install zipline 3.0.3 - issue with h5py - includes workaround

[sammmyc]

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

* Operating System: Windows 10 * Python Version: 3.10.13 * Python Bitness: 64 bit * How did you install Zipline: conda - details below * Python packages: ``` (zipline_test) C:\Users\admin>conda list alembic 1.13.1 pyhd8ed1ab_0 conda-forge appdirs 1.4.4 pyh9f0ad1d_0 conda-forge bcolz-zipline 1.2.6 py310h3e78b6c_1 conda-forge beautifulsoup4 4.12.2 pyha770c72_0 conda-forge blosc 1.21.3 h6c2663c_0 bottleneck 1.3.7 py310h3e78b6c_1 conda-forge brotli-python 1.1.0 py310h00ffb61_1 conda-forge bzip2 1.0.8 he774522_0 c-blosc2 2.12.0 h183a6f4_0 conda-forge ca-certificates 2023.12.12 haa95532_0 cached-property 1.5.2 hd8ed1ab_1 conda-forge cached_property 1.5.2 pyha770c72_1 conda-forge certifi 2023.11.17 pyhd8ed1ab_0 conda-forge charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge click 8.1.7 win_pyh7428d3b_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge empyrical-reloaded 0.5.9 pyhd8ed1ab_0 conda-forge exchange-calendars 4.2.8 pyhd8ed1ab_1 conda-forge frozendict 2.4.0 py310h8d17308_0 conda-forge greenlet 3.0.3 py310h00ffb61_0 conda-forge h5py 3.7.0 nompi_py310h00cbb18_100 conda-forge hdf5 1.12.1 h51c971a_3 html5lib 1.1 pyh9f0ad1d_0 conda-forge icc_rt 2022.1.0 h6049295_2 idna 3.6 pyhd8ed1ab_0 conda-forge importlib-metadata 7.0.1 pyha770c72_0 conda-forge importlib_resources 6.1.1 pyhd8ed1ab_0 conda-forge intervaltree 3.1.0 pyhd8ed1ab_1 conda-forge iso3166 2.1.1 pyhd8ed1ab_0 conda-forge iso4217 1.9.20220401 pyhd8ed1ab_0 conda-forge korean_lunar_calendar 0.3.1 pyhd8ed1ab_0 conda-forge libblas 3.9.0 20_win64_openblas conda-forge libcblas 3.9.0 20_win64_openblas conda-forge libffi 3.4.4 hd77b12b_0 libflang 5.0.0 h6538335_20180525 conda-forge libiconv 1.17 hcfcfb64_2 conda-forge liblapack 3.9.0 20_win64_openblas conda-forge libopenblas 0.3.25 pthreads_hc140b1d_0 conda-forge libsqlite 3.44.2 hcfcfb64_0 conda-forge libta-lib 0.4.0 hcfcfb64_2 conda-forge libxml2 2.10.4 h0ad7f3c_1 libxslt 1.1.37 h0192164_0 conda-forge llvm-meta 5.0.0 0 conda-forge lru-dict 1.3.0 py310h8d17308_0 conda-forge lxml 4.9.1 py310he2412df_0 conda-forge lz4-c 1.9.4 hcfcfb64_0 conda-forge lzo 2.10 he774522_1000 conda-forge mako 1.3.0 pyhd8ed1ab_0 conda-forge markupsafe 2.1.3 py310h8d17308_1 conda-forge multipledispatch 0.6.0 py_0 conda-forge multitasking 0.0.9 pyhd8ed1ab_0 conda-forge networkx 3.2.1 pyhd8ed1ab_0 conda-forge nomkl 1.0 h5ca1d4c_0 conda-forge numexpr 2.8.8 py310h27d5182_100 conda-forge numpy 1.26.3 py310hf667824_0 conda-forge openmp 5.0.0 vc14_1 conda-forge openssl 3.2.0 hcfcfb64_1 conda-forge packaging 23.2 pyhd8ed1ab_0 conda-forge pandas 2.1.4 py310hecd3228_0 conda-forge pandas-datareader 0.10.0 pyh6c4a22f_0 conda-forge patsy 0.5.5 pyhd8ed1ab_0 conda-forge peewee 3.17.0 py310h8b95c24_0 conda-forge pip 23.3.1 py310haa95532_0 py-cpuinfo 9.0.0 pyhd8ed1ab_0 conda-forge pyluach 2.2.0 pyhd8ed1ab_0 conda-forge pysocks 1.7.1 pyh0701188_6 conda-forge pytables 3.8.0 py310h4671533_3 python 3.10.13 he1021f5_0 python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-interface 1.6.0 py_0 conda-forge python-tzdata 2023.4 pyhd8ed1ab_0 conda-forge python_abi 3.10 2_cp310 conda-forge pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge requests 2.31.0 pyhd8ed1ab_0 conda-forge scipy 1.11.4 py310hf667824_0 conda-forge setuptools 68.2.2 py310haa95532_0 six 1.16.0 pyh6c4a22f_0 conda-forge sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge soupsieve 2.5 pyhd8ed1ab_1 conda-forge sqlalchemy 2.0.25 py310h8d17308_0 conda-forge sqlite 3.41.2 h2bbff1b_0 statsmodels 0.14.1 py310h3e78b6c_0 conda-forge ta-lib 0.4.19 py310h9b08ddd_4 conda-forge tk 8.6.12 h2bbff1b_0 toolz 0.12.0 pyhd8ed1ab_0 conda-forge typing-extensions 4.9.0 hd8ed1ab_0 conda-forge typing_extensions 4.9.0 pyha770c72_0 conda-forge tzdata 2023d h04d1e81_0 ucrt 10.0.22621.0 h57928b3_0 conda-forge urllib3 2.1.0 pyhd8ed1ab_0 conda-forge vc 14.2 h21ff451_1 vc14_runtime 14.38.33130 h82b7239_18 conda-forge vs2015_runtime 14.38.33130 hcb4865c_18 conda-forge webencodings 0.5.1 pyhd8ed1ab_2 conda-forge wheel 0.41.2 py310haa95532_0 win_inet_pton 1.1.0 pyhd8ed1ab_6 conda-forge xz 5.4.5 h8cc25b3_0 yfinance 0.2.33 pyhd8ed1ab_0 conda-forge zipline-reloaded 3.0.3 py310h9b08ddd_0 conda-forge zipp 3.17.0 pyhd8ed1ab_0 conda-forge zlib 1.2.13 h8cc25b3_0 zlib-ng 2.0.7 hcfcfb64_0 conda-forge zstd 1.5.5 hd43e919_0 ```

Now that you know a little about me, let me tell you about the issue I am having:

Description of Issue

• What did you expect to happen? I load zipline-reloaded into a fresh conda environment, and expect ziplnie to start cleanly.

• What happened instead? I run zipline and get the following error:

  File "F:\Python\anaconda3\envs\zipline_test\lib\site-packages\zipline\data\fx\__init__.py", line 4, in <module>
  from .hdf5 import HDF5FXRateReader, HDF5FXRateWriter
  File "F:\Python\anaconda3\envs\zipline_test\lib\site-packages\zipline\data\fx\hdf5.py", line 96, in <module>
  import h5py
  File "F:\Python\anaconda3\envs\zipline_test\lib\site-packages\h5py\__init__.py", line 33, in <module>
  from . import version
  File "F:\Python\anaconda3\envs\zipline_test\lib\site-packages\h5py\version.py", line 15, in <module>
  from . import h5 as _h5
  File "h5py\h5.pyx", line 1, in init h5py.h5
  ImportError: DLL load failed while importing defs: The specified procedure could not be found.

Here is how you can reproduce this issue on your machine:

Reproduction Steps

1. $ conda create -y -n zipline_test python=3.10
2. $ conda activate zipline_test
3. $ conda install -y -c conda-forge zipline-reloaded
4. $ zipline (-> look for error lines quoted above...) ...

What steps have you taken to resolve this already?

I have found a workaround.

1. $ conda create -y -n zipline_test python=3.10
2. $ conda activate zipline_test
3. $ conda install -y -c conda-forge h5py
4. $ conda install -y -c conda-forge zipline-reloaded
5. $ zipline (-> starts correctly) ...

Anything else?

I found the workaround by observing this behaviour:

1. $ conda create -y -n zipline_test python=3.10
2. $ conda activate zipline_test
3. $ conda install -y -c conda-forge zipline-reloaded
4. $ zipline (-> look for error line quoted above...)
5. $ conda uninstall h5py
6. $ conda install -y -c conda-forge h5py
7. $ conda install -y -c conda-forge zipline-reloaded
8. $ zipline (-> starts correctly) ie: remove and reload the h5py package followed by re-install of zipline-reloaded worked

Whilst this may be an obvious workaround, I thought it may be useful to capture as I did not see a specific instruction on Windows to load h5py prior to zipline-reloaded. ...

Sincerely, SamC

[gnzsnz]

using pip this requires to install debian/ubuntu package libhdf5-dev

[nucflash]

Ran into it as well. On a Mac, running py3.12 (via pyenv), you need a couple of dependencies before it installs successfully with pip:

brew install hdf5
brew install ta-lib
pip install zipline-reloaded

[stefan-jansen]

Closing for lack of activity, feel free to reopen if the issue persists.