EOFError: Ran out of input

mtomic-quant commented 4 months ago

Specifications

Problem: Bug, feature request, or other? Bug
OS: WSL2
Cvxportfolio version: 1.2.1
Python version: 3.10.11
Cvxpy version: Latest
Pandas version: 2.2.1 (tried several previous versions)
Data: Downloaded by Cvxportfolio or user-provided? user-provided

Description

All of a sudden getting EOFError: Ran out of input error when running any example, both user-provided and data downloaded by Cvxportfolio.

How to reproduce

import cvxportfolio as cvx

gamma = 3       # risk aversion parameter (Chapter 4.2)
kappa = 0.05    # covariance forecast error risk parameter (Chapter 4.3)
objective = cvx.ReturnsForecast() - gamma * (
    cvx.FullCovariance() + kappa * cvx.RiskForecastError()
) - cvx.StocksTransactionCost()
constraints = [cvx.LeverageLimit(3)]

policy = cvx.MultiPeriodOptimization(objective, constraints, planning_horizon=2)

simulator = cvx.StockMarketSimulator(['AAPL', 'AMZN', 'TSLA', 'GM', 'CVX', 'NKE'])

result = simulator.backtest(policy, start_time='2020-01-01')

Output

{
    "name": "EOFError",
    "message": "Ran out of input",
    "stack": "---------------------------------------------------------------------------
EOFError                                  Traceback (most recent call last)
Cell In[1], line 12
      8 constraints = [cvx.LeverageLimit(3)]
     10 policy = cvx.MultiPeriodOptimization(objective, constraints, planning_horizon=2)
---> 12 simulator = cvx.StockMarketSimulator(['AAPL', 'AMZN', 'TSLA', 'GM', 'CVX', 'NKE'])
     14 result = simulator.backtest(policy, start_time='2020-01-01')
     16 # print back-test result statistics

File /mnt/c/Users/tomic/open_mineral/om_data_science/nixtla/.venv/lib/python3.10/site-packages/cvxportfolio/simulator.py:745, in StockMarketSimulator.__init__(self, universe, costs, round_trades, cash_key, base_location, trading_frequency, **kwargs)
    739 def __init__(self, universe=(),
    740              costs=(StocksTransactionCost, StocksHoldingCost),
    741              round_trades=True,
    742              cash_key=\"USDOLLAR\", base_location=BASE_LOCATION,
    743              trading_frequency=None, **kwargs):
--> 745     super().__init__(universe=universe,
    746                      costs=costs, round_trades=round_trades,
    747                      cash_key=cash_key, base_location=base_location,
    748                      trading_frequency=trading_frequency, **kwargs)

File /mnt/c/Users/tomic/open_mineral/om_data_science/nixtla/.venv/lib/python3.10/site-packages/cvxportfolio/simulator.py:147, in MarketSimulator.__init__(self, universe, returns, volumes, prices, market_data, costs, round_trades, datasource, cash_key, base_location, min_history, trading_frequency)
    139         self.market_data = UserProvidedMarketData(
    140             returns=returns,
    141             volumes=volumes, prices=prices,
   (...)
    144             min_history=min_history,
    145             trading_frequency=trading_frequency)
    146     else:
--> 147         self.market_data = DownloadedMarketData(
    148             universe=universe,
    149             cash_key=cash_key,
    150             base_location=base_location,
    151             min_history=min_history,
    152             trading_frequency=trading_frequency,
    153             datasource=datasource)
    155 self.round_trades = round_trades
    156 self.costs = [el() if isinstance(el, type) else el for el in costs]

File /mnt/c/Users/tomic/open_mineral/om_data_science/nixtla/.venv/lib/python3.10/site-packages/cvxportfolio/data/market_data.py:586, in DownloadedMarketData.__init__(self, universe, datasource, cash_key, base_location, storage_backend, min_history, grace_period, trading_frequency, online_usage)
    582     self.datasource = globals()[datasource]
    583 self._get_market_data(
    584     universe, grace_period=grace_period,
    585     storage_backend=storage_backend)
--> 586 self._add_cash_column(self.cash_key, grace_period=grace_period)
    587 self._remove_missing_recent()
    589 # this is mandatory

File /mnt/c/Users/tomic/open_mineral/om_data_science/nixtla/.venv/lib/python3.10/site-packages/cvxportfolio/data/market_data.py:252, in MarketDataInMemory._add_cash_column(self, cash_key, grace_period)
    239 if self.returns.index.tz is None:
    240     raise DataError(
    241         'Your provided dataframes are not timezone aware.'
    242         + \" This is not recommended, and doesn't allow to add the cash\"
   (...)
    249         + \" prices, if provided), and set the cash_key parameter to\"
    250         + \" its name.\")
--> 252 data = Fred(
    253     RATES[cash_key], base_location=self.base_location,
    254     grace_period=grace_period)
    256 cash_returns_per_period = resample_returns(
    257     data.data/100, periods=self.periods_per_year)
    259 # we merge instead of assigning column because indexes might
    260 # be misaligned (e.g., with tz-aware timestamps)

File /mnt/c/Users/tomic/open_mineral/om_data_science/nixtla/.venv/lib/python3.10/site-packages/cvxportfolio/data/symbol_data.py:96, in SymbolData.__init__(self, symbol, storage_backend, base_location, grace_period)
     94 self._storage_backend = storage_backend
     95 self._base_location = base_location
---> 96 self.update(grace_period)
     97 self._data = self.load()

File /mnt/c/Users/tomic/open_mineral/om_data_science/nixtla/.venv/lib/python3.10/site-packages/cvxportfolio/data/symbol_data.py:184, in SymbolData.update(self, grace_period)
    169 def update(self, grace_period):
    170     \"\"\"Update current stored data for symbol.
    171 
    172     Checks (which raise warnings):
   (...)
    182     :type grace_period: pandas.Timedelta
    183     \"\"\"
--> 184     current = self._load_raw()
    185     logger.info(
    186         f\"Downloading {self.symbol}\"
    187         + f\" from {self.__class__.__name__}\")
    188     updated = self._download(
    189         self.symbol, current, grace_period=grace_period)

File /mnt/c/Users/tomic/open_mineral/om_data_science/nixtla/.venv/lib/python3.10/site-packages/cvxportfolio/data/symbol_data.py:134, in SymbolData._load_raw(self)
    129 try:
    130     logger.info(
    131         f\"{self.__class__.__name__} is trying to load {self.symbol}\"
    132         + f\" with {self._storage_backend} backend\"
    133         + f\" from {self.storage_location}\")
--> 134     return loader(self.symbol, self.storage_location)
    135 except FileNotFoundError:
    136     return None

File /mnt/c/Users/tomic/open_mineral/om_data_science/nixtla/.venv/lib/python3.10/site-packages/cvxportfolio/data/symbol_data.py:1208, in _loader_pickle(symbol, storage_location)
   1206 def _loader_pickle(symbol, storage_location):
   1207     \"\"\"Load data in pickle format.\"\"\"
-> 1208     return pd.read_pickle(storage_location / f\"{symbol}.pickle\")

File /mnt/c/Users/tomic/open_mineral/om_data_science/nixtla/.venv/lib/python3.10/site-packages/pandas/io/pickle.py:208, in read_pickle(filepath_or_buffer, compression, storage_options)
    205     with warnings.catch_warnings(record=True):
    206         # We want to silence any warnings about, e.g. moved modules.
    207         warnings.simplefilter(\"ignore\", Warning)
--> 208         return pickle.load(handles.handle)
    209 except excs_to_catch:
    210     # e.g.
    211     #  \"No module named 'pandas.core.sparse.series'\"
    212     #  \"Can't get attribute '__nat_unpickle' on <module 'pandas._libs.tslib\"
    213     return pc.load(handles.handle, encoding=None)

EOFError: Ran out of input"
}

enzbus commented 4 months ago

Short answer: delete the ~/cvxportfolio_data folder and re-run.Long answer: I need to look more closely at the stack trace.Let me know if it works!

mtomic-quant commented 4 months ago

Short answer: delete the ~/cvxportfolio_data folder and re-run.Long answer: I need to look more closely at the stack trace.Let me know if it works!

works now, thanks!

enzbus commented 4 months ago

Should be fixed now (in master, will be released sometimes before EOM). Looks like you had quit a Cvxportfolio program by killing the process while it was updating market data files, and a DFF.pickle file (for the US central bank rates) remained half written. Now that exception is caught and the file discarded automatically (but you still get a warning).

mtomic-quant commented 4 months ago

Should be fixed now (in master, will be released sometimes before EOM). Looks like you had quit a Cvxportfolio program by killing the process while it was updating market data files, and a DFF.pickle file (for the US central bank rates) remained half written. Now that exception is caught and the file discarded automatically (but you still get a warning).

Thank you, appreciate it

cvxgrp / cvxportfolio