IndexError: index -1 is out of bounds for axis 0 with size 0

[polakowo]

Problem Description

I already tried lots of different returns to create a full tear sheet but still cannot get it working, while simple tear sheet works.

import yfinance as yf
import pyfolio as pf

fb_history = yf.Ticker("FB").history()
rets = fb_history['Close'].pct_change().dropna()
pf.create_full_tear_sheet(rets)

IndexError                                Traceback (most recent call last)
<ipython-input-1-b05f97dd412a> in <module>
      4 fb_history = yf.Ticker("FB").history()
      5 rets = fb_history['Close'].pct_change().dropna()
----> 6 pf.create_full_tear_sheet(rets)

~/miniconda3/lib/python3.7/site-packages/pyfolio/tears.py in create_full_tear_sheet(returns, positions, transactions, market_data, benchmark_rets, slippage, live_start_date, sector_mappings, bayesian, round_trips, estimate_intraday, hide_positions, cone_std, bootstrap, unadjusted_returns, style_factor_panel, sectors, caps, shares_held, volumes, percentile, turnover_denom, set_context, factor_returns, factor_loadings, pos_in_dollars, header_rows, factor_partitions)
    209         turnover_denom=turnover_denom,
    210         header_rows=header_rows,
--> 211         set_context=set_context)
    212 
    213     create_interesting_times_tear_sheet(returns,

~/miniconda3/lib/python3.7/site-packages/pyfolio/plotting.py in call_w_context(*args, **kwargs)
     50         if set_context:
     51             with plotting_context(), axes_style():
---> 52                 return func(*args, **kwargs)
     53         else:
     54             return func(*args, **kwargs)

~/miniconda3/lib/python3.7/site-packages/pyfolio/tears.py in create_returns_tear_sheet(returns, positions, transactions, live_start_date, cone_std, benchmark_rets, bootstrap, turnover_denom, header_rows, return_fig)
    502                              header_rows=header_rows)
    503 
--> 504     plotting.show_worst_drawdown_periods(returns)
    505 
    506     vertical_sections = 11

~/miniconda3/lib/python3.7/site-packages/pyfolio/plotting.py in show_worst_drawdown_periods(returns, top)
   1662     """
   1663 
-> 1664     drawdown_df = timeseries.gen_drawdown_table(returns, top=top)
   1665     utils.print_table(
   1666         drawdown_df.sort_values('Net drawdown in %', ascending=False),

~/miniconda3/lib/python3.7/site-packages/pyfolio/timeseries.py in gen_drawdown_table(returns, top)
    989 
    990     df_cum = ep.cum_returns(returns, 1.0)
--> 991     drawdown_periods = get_top_drawdowns(returns, top=top)
    992     df_drawdowns = pd.DataFrame(index=list(range(top)),
    993                                 columns=['Net drawdown in %',

~/miniconda3/lib/python3.7/site-packages/pyfolio/timeseries.py in get_top_drawdowns(returns, top)
    954     drawdowns = []
    955     for t in range(top):
--> 956         peak, valley, recovery = get_max_drawdown_underwater(underwater)
    957         # Slice out draw-down period
    958         if not pd.isnull(recovery):

~/miniconda3/lib/python3.7/site-packages/pyfolio/timeseries.py in get_max_drawdown_underwater(underwater)
    893     valley = np.argmin(underwater)  # end of the period
    894     # Find first 0
--> 895     peak = underwater[:valley][underwater[:valley] == 0].index[-1]
    896     # Find last 0
    897     try:

~/miniconda3/lib/python3.7/site-packages/pandas/core/indexes/extension.py in __getitem__(self, key)
    213 
    214     def __getitem__(self, key):
--> 215         result = self._data[key]
    216         if isinstance(result, type(self._data)):
    217             if result.ndim == 1:

~/miniconda3/lib/python3.7/site-packages/pandas/core/arrays/datetimelike.py in __getitem__(self, key)
    536         if lib.is_integer(key):
    537             # fast-path
--> 538             result = self._data[key]
    539             if self.ndim == 1:
    540                 return self._box_func(result)

IndexError: index -1 is out of bounds for axis 0 with size 0

Versions

• Pyfolio version: 0.9.2
• Python version: 3.7.3
• Pandas version: 1.1.3
• Matplotlib version: 3.1.0

[hirenpatel101]

Hey man, I had the same problem. The way I solved this is by changing this line in timeseries.py within the actual package:

893     valley = np.argmin(underwater)  # end of the period

to:

893     valley = underwater.index[np.argmin(underwater)]  # end of the period

I think this is because valley is expected to be a datetime (at least that's what it says in the docs above it), but the actual result that comes from np.argmin(underwater) is an integer (the index of the drawdown valley). Using this integer to then get the actual datetime allows the function to continue and produce a full tear sheet. No idea if the drawdown numbers are correct, as I'm quite new to trading myself and all the terms, but looks correct

Hope this helps

[polakowo]

Thanks. It surprises me that this bug hasn't been fixed yet. Seems like this package isn't actively maintained anymore.

[cactus1549]

@polakowo Yes, this code (and zipline) doesn't appear to be very actively maintained. FYI, with the following (somewhat outdated) versions:

• python 3.6.12
• pyfolio 0.8.0
• pandas 0.22.0
• matplotlib 3.0.3
• numpy 1.19.5 (which are basically the versions to make zipline 1.4.1 run more or less correctly) this particular error does not appear because np.argmin(underwater) does indeed return a Datetime.

[polakowo]

@cactus1549 I wanted to integrate pyfolio into vectorbt but realized it wasn't worth the effort of adding a library that uses an outdated tech stack. Just implemented the most useful plots from scratch + added interactivity with Plotly.

[cactus1549]

Yes, @polakowo I think you made the correct decision. I initially thought of using zipline, but was too frustrated by the lack of support. I ended up writing my own ad-hoc backtester but adopting their performance dataframe format thinking I'd be able to use all the analysis tools such as pyfolio, but that's proven a bit of a pain also. Does vectorbt have analysis tools? Would it be relatively easy to switch over?

Also @hirenpatel101, in case anyone wants to continue with pyfolio, I think the future-proof code fix for that line should be:

valley = underwater.index[underwater.values.argmin()]

[polakowo]

@cactus1549 vectorbt integrates backtesting and data analysis so it has tons of cool stuff. It doesn't offer all the plots from pyfolio, but it's definitely more flexible in computing/visualizing custom metrics. For example, plotting the distribution of trade returns is as simple as portfolio.trades.returns.histplot(), where both trades and trades.returns consist of numpy arrays, which can be analyzed with other tools.

The complexity of switching over depends upon the use case - vectorbt doesn't implement margin trading and futures yet, neither it implements some complex order management (It ain't a native backtester per se that competes with backtrader and zipline). But as soon as you can represent your orders in an array format, it's very much superior in terms of performance and analysis tools. What has worked for me in the past is prototyping my strategy with vectorbt, doing a final test with another backtester, deploying the strategy, and then again analyzing its performance with vectorbt.

[cactus1549]

@polakowo, thank you, it does sound interesting. I'll check it out.