bug: Incorrect computation of metrics when using group_by

[andreas-vester]

If I group_by using the Portfolio.stats() method, there might be a bug.

Consider the following example:

import pandas as pd

import vectorbt as vbt
from datetime import datetime

symbols = ["SPY", "QQQ", "DBC"]
start=datetime(2020,12,30)
end=datetime(2023,10,31)

prices = vbt.YFData.download(symbols=symbols, start=start, end=end, tz_localize=None)
close = prices.get("Close")

# compute strategy
sma = vbt.MA.run(close, 10)
entries = sma.close_crossed_above(sma.ma)
exits = sma.close_crossed_below(sma.ma)

# create portfolio object
pf = vbt.Portfolio.from_signals(close, entries, exits)

group_by = ["equity", "equity", "comdty"]

# The following calls should yield the same metrics, such as maximum drawdown etc.  

print(pf.stats(agg_func=None, group_by=group_by).T)  # here, max drawdown (and others) might be incorrect)

                                               equity                     comdty
Start                       2020-12-29 05:00:00+00:00  2020-12-29 05:00:00+00:00
End                         2023-10-30 04:00:00+00:00  2023-10-30 04:00:00+00:00
Period                                            714                        714
Start Value                                     200.0                      100.0
End Value                                  191.556918                 129.202912
Total Return [%]                            -4.221541                  29.202912
Benchmark Return [%]                        14.963537                  70.093006
Max Gross Exposure [%]                          100.0                      100.0
Total Fees Paid                                   0.0                        0.0
Max Drawdown [%]                            25.921027                  22.123985
Max Drawdown Duration                           497.0                      414.0
Total Trades                                      122                         57
Total Closed Trades                               122                         57
Total Open Trades                                   0                          0
Open Trade PnL                                    0.0                        0.0
Win Rate [%]                                 34.42623                  31.578947
Best Trade [%]                              12.433743                  25.701623
Worst Trade [%]                             -5.036392                  -6.314305
Avg Winning Trade [%]                        3.005626                   5.095227
Avg Losing Trade [%]                        -1.618623                   -1.63749
Avg Winning Trade Duration                  13.642857                  15.888889
Avg Losing Trade Duration                      2.9375                   3.081081
Profit Factor                                0.934748                   1.383952
Expectancy                                  -0.069206                   0.440204

print(pf.stats(agg_func=None).groupby(group_by, sort=False).mean().T)  # here, max drawdown seems reasonable

                                               equity                     comdty
Start                       2020-12-29 05:00:00+00:00  2020-12-29 05:00:00+00:00
End                         2023-10-30 04:00:00+00:00  2023-10-30 04:00:00+00:00
Period                                          714.0                      714.0
Start Value                                     100.0                      100.0
End Value                                   95.778459                 129.202912
Total Return [%]                            -4.221541                  29.202912
Benchmark Return [%]                        14.963537                  70.093006
Max Gross Exposure [%]                          100.0                      100.0
Total Fees Paid                                   0.0                        0.0
Max Drawdown [%]                            26.373835                  22.123985
Max Drawdown Duration                           443.0                      414.0
Total Trades                                     61.0                       57.0
Total Closed Trades                              61.0                       57.0
Total Open Trades                                 0.0                        0.0
Open Trade PnL                                    0.0                        0.0
Win Rate [%]                                34.463277                  31.578947
Best Trade [%]                              10.024199                  25.701623
Worst Trade [%]                             -4.533775                  -6.314305
Avg Winning Trade [%]                        3.005626                   5.095227
Avg Losing Trade [%]                        -1.633423                   -1.63749
Avg Winning Trade Duration                  13.642857                  15.888889
Avg Losing Trade Duration                    2.935464                   3.081081
Profit Factor                                0.929021                   1.383952
Expectancy                                  -0.068112                   0.440204

I am focussing on max. drawdowns. If I compute the average max. drawdown based on pf.drawdown().min() for the equity group_by, i.e. avg(SPY,QQQ), I get the same result as compared to calling pf.stats(agg_func=None).groupby(group_by, sort=False).mean().T.

print(pf.drawdown().min())

ma_window  symbol
10         SPY      -0.191162
           QQQ      -0.336314
           DBC      -0.221240
dtype: float64

avg_dd_equity = pf.drawdown().min().iloc[:2].mean()
print(f"Avg. Drawdown group 'equity' (SPY and QQQ): {avg_dd_equity}")

Avg. Drawdown group 'equity' (SPY and QQQ): -0.26373835274077756

There's a different max. drawdown when calling pf.stats(agg_func=None, group_by=group_by).T.

Am I missing something? For instance, total return is the same for both calls. Is this a bug?

[polakowo]

Not a bug, you cannot expect the max drawdown of a grouped portfolio (where equities are added together) be the same as the average max drawdown of individual portfolios. If one equity has a max drawdown of 10% and another one has also a 10% max drawdown, and both drawdowns do not overlap in time, then the max drawdown of the combined portfolio would be 5% and not 10%.

[andreas-vester]

Ok, fair point. I probably misunderstood the concept of the input argument group_by in Portfolio.

Are the following statements correct?

• pf.stats(agg_func=None, group_by=["equity", "equity", "comdty"])
  This will put the two equity tickers into one portfolio, but they don't share their cash. They are basically two independent tickers applying the same strategy. The final portfolio equity curve can be computed by taking the mean of the daily returns of both individual tickers. The stats then belong to the portfolio (i.e. combined equity curve).
• pf.stats(agg_func=None, group_by=["equity", "equity", "comdty"], cash_sharing=True)
  This will put the two equity tickers into one portfolio, and they share their cash (i.e. starting equity value of the portfolio is 100 rather than 200). They are basically two independent tickers applying the same strategy. For cash sharing, you probably need to (periodically) specify the weights each ticker gets. What would happen if you don't specify weights but only provide entry and exit signals for both tickers? Would the first ticker with a long signal take 100% of the available cash? And would any subsequent long signal in another ticker be ignored because there's no cash left? The final portfolio equity curve can be computed by taking the weighted average of the daily returns of both individual tickers. The stats then belong to the portfolio (i.e. combined equity curve).
• pf.stats(agg_func=None).groupby(group_by, sort=False).mean()
  This really treats every ticker as an individual portfolio. The average of the stats is really the average of individual portfolios.
  This approach seems to be appropriate if I want to apply a trading algo to many different time series in order to evaluate what I can expect on average from the strategy itself.

[andreas-vester]

@polakowo Short feedback would highly be appreciated. Thanks a lot!