search

stefan-jansen / zipline-reloaded

Zipline, a Pythonic Algorithmic Trading Library
https://zipline.ml4trading.io
Apache License 2.0
1.03k stars 199 forks source link

support for 3.12 #241

Open gnzsnz opened 3 months ago

gnzsnz commented 3 months ago

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

* Operating System: x86_64 GNU/Linux & aarch64 GNU/Linux * Python Version: `$ python --version` --> 3.12 * Python Bitness: `$ python -c 'import math, sys;print(int(math.log(sys.maxsize + 1, 2) + 1))'` * How did you install Zipline: (`pip`, `conda`, or `other (please explain)`) pip * Python packages: `$ pip freeze` or `$ conda list`

Now that you know a little about me, let me tell you about the issue I am having:

Description of Issue

run pip install zipline-reloaded on python 3.12

instalation fails

Here is how you can reproduce this issue on your machine:

Reproduction Steps

  1. docker run --rm -it python:3.12 bash
apt-get update && apt-get update && apt-get install --no-install-recommends -y libhdf5-dev
pip install git+https://github.com/stefan-jansen/bcolz-zipline.git@main
curl -LO https://github.com/gnzsnz/jupyter-quant/releases/download/ta-lib-0.4.0-linux/ta-lib-0.4.0-linux_"$(uname -m)".tgz
tar xzf ta-lib-0.4.0-linux_x86_64.tgz
export PREFIX=/usr/local/ta-lib &&   export TA_LIBRARY_PATH="$PREFIX/lib" &&   export TA_INCLUDE_PATH="$PREFIX/include"
pip install zipline-reloaded

What steps have you taken to resolve this already?

install bcolz from git helps to solve the problems with bcolz and 3.12. but zipline still fails

Anything else?

log

     creating build/temp.linux-x86_64-cpython-312
      creating build/temp.linux-x86_64-cpython-312/src
      creating build/temp.linux-x86_64-cpython-312/src/zipline
      creating build/temp.linux-x86_64-cpython-312/src/zipline/assets
      gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/tmp/pip-build-env-z5lzvvey/overlay/lib/python3.12/site-packages/numpy/core/include
-I/usr/local/include/python3.12 -c src/zipline/assets/_assets.c -o build/temp.linux-x86_64-cpython-312/src/zipline/assets/_assets.o
      src/zipline/assets/_assets.c: In function ‘__pyx_pf_7zipline_6assets_7_assets_5Asset___init__’:
      src/zipline/assets/_assets.c:1519:32: error: ‘_PyCFrame’ has no member named ‘use_tracing’
       1519 |      (unlikely((tstate)->cframe->use_tracing) &&\
            |                                ^~
      src/zipline/assets/_assets.c:968:43: note: in definition of macro ‘unlikely’
        968 |   #define unlikely(x) __builtin_expect(!!(x), 0)
            |                                           ^
      src/zipline/assets/_assets.c:1558:15: note: in expansion of macro ‘__Pyx_IsTracing’
       1558 |           if (__Pyx_IsTracing(tstate, 1, 1)) {\
            |               ^~~~~~~~~~~~~~~
      src/zipline/assets/_assets.c:2685:3: note: in expansion of macro ‘__Pyx_TraceCall’
       2685 |   __Pyx_TraceCall("__init__", __pyx_f[0], 93, 0, __PYX_ERR(0, 93, __pyx_L1_error));
            |   ^~~~~~~~~~~~~~~
      src/zipline/assets/_assets.c:1519:32: error: ‘_PyCFrame’ has no member named ‘use_tracing’
       1519 |      (unlikely((tstate)->cframe->use_tracing) &&\
            |                                ^~
      src/zipline/assets/_assets.c:968:43: note: in definition of macro ‘unlikely’
        968 |   #define unlikely(x) __builtin_expect(!!(x), 0)
            |                                           ^
      src/zipline/assets/_assets.c:1566:11: note: in expansion of macro ‘__Pyx_IsTracing’
       1566 |       if (__Pyx_IsTracing(tstate, 1, 1)) {\
            |           ^~~~~~~~~~~~~~~
      src/zipline/assets/_assets.c:2685:3: note: in expansion of macro ‘__Pyx_TraceCall’
       2685 |   __Pyx_TraceCall("__init__", __pyx_f[0], 93, 0, __PYX_ERR(0, 93, __pyx_L1_error));
            |   ^~~~~~~~~~~~~~~
      src/zipline/assets/_assets.c:1519:32: error: ‘_PyCFrame’ has no member named ‘use_tracing’
       1519 |      (unlikely((tstate)->cframe->use_tracing) &&\
            |                                ^~
      src/zipline/assets/_assets.c:968:43: note: in definition of macro ‘unlikely’
        968 |   #define unlikely(x) __builtin_expect(!!(x), 0)
            |                                           ^
      src/zipline/assets/_assets.c:1617:19: note: in expansion of macro ‘__Pyx_IsTracing’
       1617 |               if (__Pyx_IsTracing(tstate, 0, 0)) {\
            |                   ^~~~~~~~~~~~~~~
      src/zipline/assets/_assets.c:2824:3: note: in expansion of macro ‘__Pyx_TraceReturn’
       2824 |   __Pyx_TraceReturn(Py_None, 0);
            |   ^~~~~~~~~~~~~~~~~
      src/zipline/assets/_assets.c:1519:32: error: ‘_PyCFrame’ has no member named ‘use_tracing’
       1519 |      (unlikely((tstate)->cframe->use_tracing) &&\
            |                                ^~

logs is very long, but the same error repeats over and over. Pyx_TraceCall, __Pyx_IsTracing and Pyx_TraceReturn give errors.

Sincerely, $ whoami

RichardDale commented 2 months ago

I think Zipline-Released v3.0.4 resolves this issue. Please confirm!

gnzsnz commented 2 months ago

i'm testing this on 3.12.3 on ubuntu 22.04 and osx Sonoma

================================================================================================== ERRORS ==================================================================================================
_____________________________________________________________________________ ERROR collecting tests/data/bundles/test_core.py _____________________________________________________________________________
ImportError while importing test module '/home/gonzo/Documents/zipline-reloaded/tests/data/bundles/test_core.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../.pyenv/versions/3.12.3/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/data/bundles/test_core.py:5: in <module>
    from parameterized import parameterized
E   ModuleNotFoundError: No module named 'parameterized'
____________________________________________________________________________ ERROR collecting tests/data/bundles/test_csvdir.py ____________________________________________________________________________
ImportError while importing test module '/home/gonzo/Documents/zipline-reloaded/tests/data/bundles/test_csvdir.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../.pyenv/versions/3.12.3/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/data/bundles/test_csvdir.py:14: in <module>
    from zipline.testing.github_actions import skip_on
../../.pyenv/versions/3.12.3/envs/zipline/lib/python3.12/site-packages/zipline/testing/__init__.py:1: in <module>
    from .core import (  # noqa
../../.pyenv/versions/3.12.3/envs/zipline/lib/python3.12/site-packages/zipline/testing/core.py:19: in <module>
    from testfixtures import TempDirectory
E   ModuleNotFoundError: No module named 'testfixtures'
____________________________________________________________________________ ERROR collecting tests/data/bundles/test_quandl.py ____________________________________________________________________________
ImportError while importing test module '/home/gonzo/Documents/zipline-reloaded/tests/data/bundles/test_quandl.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../.pyenv/versions/3.12.3/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/data/bundles/test_quandl.py:14: in <module>
    from zipline.testing import (
../../.pyenv/versions/3.12.3/envs/zipline/lib/python3.12/site-packages/zipline/testing/__init__.py:1: in <module>
    from .core import (  # noqa
../../.pyenv/versions/3.12.3/envs/zipline/lib/python3.12/site-packages/zipline/testing/core.py:19: in <module>
    from testfixtures import TempDirectory
E   ModuleNotFoundError: No module named 'testfixtures'
_____________________________________________________________________________ ERROR collecting tests/data/test_adjustments.py ______________________________________________________________________________

on both environments I'm getting the error above when i run pytest test

the error repeats for the following test modules

========================================================================================= short test summary info ==========================================================================================
ERROR tests/data/bundles/test_core.py
ERROR tests/data/bundles/test_csvdir.py
ERROR tests/data/bundles/test_quandl.py
ERROR tests/data/test_adjustments.py
ERROR tests/data/test_daily_bars.py
ERROR tests/data/test_dispatch_bar_reader.py
ERROR tests/data/test_fx.py
ERROR tests/data/test_hdf5_daily_bars.py
ERROR tests/data/test_minute_bars.py
ERROR tests/data/test_resample.py
ERROR tests/events/test_events.py
ERROR tests/events/test_events_cme.py
ERROR tests/events/test_events_nyse.py
ERROR tests/finance/test_commissions.py
ERROR tests/finance/test_slippage.py
ERROR tests/metrics/test_core.py
ERROR tests/metrics/test_metrics.py
ERROR tests/pipeline/test_adjusted_array.py
ERROR tests/pipeline/test_alias.py
ERROR tests/pipeline/test_classifier.py
ERROR tests/pipeline/test_column.py
ERROR tests/pipeline/test_computable_term.py
ERROR tests/pipeline/test_domain.py
ERROR tests/pipeline/test_downsampling.py
ERROR tests/pipeline/test_dtypes.py
ERROR tests/pipeline/test_engine.py
ERROR tests/pipeline/test_events.py
ERROR tests/pipeline/test_factor.py
ERROR tests/pipeline/test_filter.py
ERROR tests/pipeline/test_hooks.py
ERROR tests/pipeline/test_international_markets.py
ERROR tests/pipeline/test_numerical_expression.py
ERROR tests/pipeline/test_pipeline_algo.py
ERROR tests/pipeline/test_quarters_estimates.py
ERROR tests/pipeline/test_slice.py
ERROR tests/pipeline/test_statistical.py
ERROR tests/pipeline/test_technical.py
ERROR tests/pipeline/test_term.py
ERROR tests/pipeline/test_us_equity_pricing_loader.py
ERROR tests/test_algorithm.py
ERROR tests/test_api_shim.py
ERROR tests/test_assets.py
ERROR tests/test_bar_data.py
ERROR tests/test_benchmark.py
ERROR tests/test_blotter.py
ERROR tests/test_cmdline.py
ERROR tests/test_continuous_futures.py
ERROR tests/test_data_portal.py
ERROR tests/test_examples.py
ERROR tests/test_execution_styles.py
ERROR tests/test_fetcher.py
ERROR tests/test_finance.py
ERROR tests/test_history.py
ERROR tests/test_labelarray.py
ERROR tests/test_ordering.py
ERROR tests/test_restrictions.py
ERROR tests/test_security_list.py
ERROR tests/test_testing.py
ERROR tests/test_tradesimulation.py
ERROR tests/utils/test_pandas_utils.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 60 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================================================================== 110 warnings, 60 errors in 3.96s =====================================================================================
yxie80 commented 1 week ago

This error shows up when testing package with python 3.12 with following command from zipline.api import order_target, record, symbol Screenshot 2024-07-07 at 10 54 40 PM