search

robertmartin8 / PyPortfolioOpt

Financial portfolio optimisation in python, including classical efficient frontier, Black-Litterman, Hierarchical Risk Parity
https://pyportfolioopt.readthedocs.io/
MIT License
4.24k stars 927 forks source link

RuntimeError: can't register atexit after shutdown when calling S = CovarianceShrinkage(df).ledoit_wolf() #558

Closed c0indev3l closed 7 months ago

c0indev3l commented 10 months ago

Hello,

I'm running on a Linux VPS (Debian Bookworm) a trading strategy as papertrade with Blankly Finance and PyPortfolioOpt. I'm using poetry to work with Python virtual environment.

$ poetry --version
Poetry (version 1.5.1)
$ poetry run python --version
Python 3.11.2

Unfortunately I'm getting a RuntimeError: can't register atexit after shutdown when calling S = CovarianceShrinkage(df_bars_not_null).ledoit_wolf()

I never experienced this kind of error in backtest. Only when trying to papertrade

  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/blankly/frameworks/strategy/strategy.py",
line 127, in rest_event
    callback(*args)
  File "/home/debian/github/scls19fr/crypto-trading-bots/blankly/multisymbols/bot_core.py", line 204, in daily_bar_event
    return multibar_daily_event(state.variables.daily_bars, state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/debian/github/scls19fr/crypto-trading-bots/blankly/multisymbols/bot_core.py", line 94, in multibar_daily_event
    S = CovarianceShrinkage(df_bars_not_null).ledoit_wolf()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/pypfopt/risk_models.py", line 384, in __in
it__
    from sklearn import covariance
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/sklearn/__init__.py", line 83, in <module>
    from .base import clone
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/sklearn/base.py", line 19, in <module>
    from .utils import _IS_32BIT
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/sklearn/utils/__init__.py", line 19, in <m
odule>
    from . import _joblib, metadata_routing
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/sklearn/utils/_joblib.py", line 7, in <mod
ule>
    import joblib
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/joblib/__init__.py", line 129, in <module>
    from .parallel import Parallel
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/joblib/parallel.py", line 31, in <module>
    from ._parallel_backends import (FallbackToBackend, MultiprocessingBackend,
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/joblib/_parallel_backends.py", line 16, in <module>
    from .pool import MemmappingPool
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/joblib/pool.py", line 31, in <module>
    from ._memmapping_reducer import get_memmapping_reducers
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/joblib/_memmapping_reducer.py", line 37, in <module>
    from .externals.loky.backend import resource_tracker
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/joblib/externals/loky/__init__.py", line 18, in <module>
    from .backend.context import cpu_count
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/joblib/externals/loky/backend/__init__.py", line 4, in <module>
    from .context import get_context
  File "/home/debian/.cache/pypoetry/virtualenvs/multisymbols-37LxTeu7-py3.11/lib/python3.11/site-packages/joblib/externals/loky/backend/context.py", line 28, in <module>
    from concurrent.futures.process import _MAX_WINDOWS_WORKERS
  File "/usr/lib/python3.11/concurrent/futures/process.py", line 101, in <module>
    threading._register_atexit(_python_exit)
  File "/usr/lib/python3.11/threading.py", line 1520, in _register_atexit
    raise RuntimeError("can't register atexit after shutdown")
RuntimeError: can't register atexit after shutdown

Do you have any idea what can cause this kind of issue?

Kind regards

88d52bdba0366127fffca9dfa93895 commented 9 months ago

Sorry but this error is strange and hard to reproduce, I think your error is here from sklearn import covariance, it from sklearn, could you try to import that and let see is there any error from that.