portfolio.stats different from select_metric [RSI 30/70 SL 0.03]

[jacopomaroli]

hey there! First of all let me congratulate with you for this amazing project. HW accelerated backtesting is amazing :D

I'm working on a comparison between a simple strategy I'm using (RSI 30 buy, RSI 70 sell with 0.03 stop loss) vs the one based on DMAC you proposed here https://nbviewer.jupyter.org/github/polakowo/vectorbt/blob/master/examples/BitcoinDMAC.ipynb

here's the gist (you can open it in colab as well): https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/75269fb9db0511ccfede1032148e9e6e/raw/b6d8f2f5857ba52ac473814252845a66ae3c82e3/strategy.ipynb

when I get the stats from the dashboard at the bottom I get

Total return    1.367398
Win rate    0.272727
Expectancy  12.430895
Max drawdown    -0.230855

but on the report in the middle of the notebook when I trigger portfolio.stats() I get

Start                     2018-01-01 00:00:00
End                       2020-01-01 00:00:00
Duration                    731 days 00:00:00
Init. Cash                                100
Total Profit                         -61.0304
Total Return [%]                     -61.0304
Benchmark Return [%]                 -47.2793
Position Coverage [%]                 49.3844
Max. Drawdown [%]                     72.7948
Avg. Drawdown [%]                     36.6465
Max. Drawdown Duration      725 days 00:00:00
Avg. Drawdown Duration      363 days 12:00:00
Num. Trades                                 8
Win Rate [%]                             37.5
Best Trade [%]                        10.6714
Worst Trade [%]                      -37.8141
Avg. Trade [%]                       -9.74853
Max. Trade Duration         130 days 00:00:00
Avg. Trade Duration          45 days 03:00:00
Expectancy                            -7.6288
SQN                                  -1.76052
Gross Exposure                       0.493844
Sharpe Ratio                        -0.582461
Sortino Ratio                        -0.78063
Calmar Ratio                        -0.515616

I tend to believe the first of the two as I simply have difficulties to accept such a simple strategy could be that bad

I think there might be conceptual problems here and there as I'm new to this lib. If you think it's an interesting notebook and would like to add it to the examples I'm happy to polish it a bit and submit a PR. It would be also great just knowing if I'm doing it right or wrong as I don't have much of a feedback

Thanks!

[polakowo]

Hi, you may want to try running without stop loss and see how it works. Not sure what del_adj method does, but there is no need to remove adjacent signals explicitly since Portfolio.from_signals executes the first one and ignores the rest automatically (unless accumulate is set to True). But this won't work with stop loss because it requires cleaned (no adjacent entries or exits) signals as input. So what you have to do is, before generating stop-loss exits, to run entries.vbt.signals.first(reset_by=exits, allow_gaps=True) and exits.vbt.signals.first(reset_by=entries, allow_gaps=True).

[jacopomaroli]

Hey thanks for getting back. I took some time to make a couple of tests today.

without SL https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/7ef01a0e11b7b8df2596201066dda4d2/raw/013cda3e7fded24715eb61569085a016cb6c3942/strategy-nosl.ipynb

Start                     2018-01-01 00:00:00
End                       2020-01-01 00:00:00
Duration                    731 days 00:00:00
Init. Cash                                100
Total Profit                         -67.3491
Total Return [%]                     -67.3491
Benchmark Return [%]                 -47.2793
Position Coverage [%]                  55.814
Max. Drawdown [%]                     74.1305
Avg. Drawdown [%]                     37.3143
Max. Drawdown Duration      725 days 00:00:00
Avg. Drawdown Duration      363 days 12:00:00
Num. Trades                                 8
Win Rate [%]                             37.5
Best Trade [%]                        10.6714
Worst Trade [%]                      -37.8141
Avg. Trade [%]                       -9.74853
Max. Trade Duration         130 days 00:00:00
Avg. Trade Duration          45 days 03:00:00
Expectancy                            -7.6288
SQN                                  -1.76052
Gross Exposure                        0.55814
Sharpe Ratio                        -0.711903
Sortino Ratio                       -0.957506
Calmar Ratio                        -0.577566
dtype: object

SL + suggested fixes to remove adjacent entries exists (should I manually trim exits before entries and entries after last exit?) https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/0795d9776c28d34ebbfc8ce5f3658b17/raw/846c44d0a1704d81c7b22c26c33aefef06b0d968/strategy-fixes.ipynb

Start                     2018-01-01 00:00:00
End                       2020-01-01 00:00:00
Duration                    731 days 00:00:00
Init. Cash                                100
Total Profit                          -43.383
Total Return [%]                      -43.383
Benchmark Return [%]                 -47.2793
Position Coverage [%]                 9.71272
Max. Drawdown [%]                     51.3615
Avg. Drawdown [%]                     25.9298
Max. Drawdown Duration      725 days 00:00:00
Avg. Drawdown Duration      363 days 12:00:00
Num. Trades                                 9
Win Rate [%]                          22.2222
Best Trade [%]                        4.60953
Worst Trade [%]                      -15.2868
Avg. Trade [%]                        -5.9443
Max. Trade Duration          33 days 00:00:00
Avg. Trade Duration           7 days 21:20:00
Expectancy                           -4.82034
SQN                                  -2.73978
Gross Exposure                      0.0971272
Sharpe Ratio                         -1.23733
Sortino Ratio                        -1.44574
Calmar Ratio                        -0.481421
dtype: object

last week 1m timeframe (no SL) https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/34f86f8a95ed31fdb4f3d9bd46822112/raw/8350b053269631fa061916611b4671d9a3805a98/strategy-cmp.ipynb

Start                     2021-03-04 00:00:00+00:00
End                       2021-03-10 23:58:00+00:00
Duration                            6 days 06:31:00
Init. Cash                                      100
Total Profit                               -81.2357
Total Return [%]                           -81.2357
Benchmark Return [%]                        10.8492
Position Coverage [%]                       46.5397
Max. Drawdown [%]                           81.2357
Avg. Drawdown [%]                           81.2357
Max. Drawdown Duration              6 days 06:17:00
Avg. Drawdown Duration              6 days 06:17:00
Num. Trades                                     160
Win Rate [%]                                      0
Best Trade [%]                            -0.273555
Worst Trade [%]                            -4.48428
Avg. Trade [%]                             -1.03027
Max. Trade Duration                 0 days 02:11:00
Avg. Trade Duration          0 days 00:25:59.625000
Expectancy                                -0.505696
SQN                                        -12.2855
Gross Exposure                             0.465397
Sharpe Ratio                               -126.723
Sortino Ratio                              -132.686
Calmar Ratio                               -1.23099
dtype: object

Last week 1m timeframe (no SL) from my current backtesting suit (slightly adapted from here https://github.com/rodrigo-brito/backtrader-binance-bot to fetch data from binance and performing only RSI 30/70 without stop loss and EMA)

Final Portfolio Value: 105.54
Profit 5.544%
Trade Analysis Results:
               Total Open     Total Closed   Total Won      Total Lost
               1              29             18             11
               Strike Rate    Win Streak     Losing Streak  PnL Net
               1              5              3              5.86
SQN: 0.84

I have a doubt: are the moving indicators in vectorbt considering the shifting in time? e.g. If I travel back in time with a graph to 1 week ago I see RSI to let's say 70 but now it's 40 because it normalized across a different range during last week. same for MACD, EMA etc... I see here https://nbviewer.jupyter.org/github/polakowo/vectorbt/blob/master/examples/MACDVolume.ipynb you're just applying MACD on the data, so I assumed the value you get from the indicator are the values at that point in time and not relative to now. It's confusing but I hope you get what I meant. If not let me know and I'll try and explain myself better maybe with a couple of pictures.

Thanks!

[polakowo]

RSI is based on rolling windows, so each generated value is relative to the rolling window preceding it. vbt.RSI gives the same results as both talib and ta packages. The window you pass to the indicator must have the same frequency as data, that is, a window size of 10 for the 1-minute timeframe will roll a 10-minute window across your data, leading to lots of fluctuations in indicator value. So you can't just switch timeframes, but need to adapt the parameters as well.

The issue with the 1-minute timeframe is that you set pretty high fees and slippage at the beginning of the notebook (0.0025 = 0.25%), and so each of your trades results in a loss because they are too frequent and cannot ride a price movement that would at least beat the cost of the trade.

You can further analyze your orders by calling portfolio.orders.records_readable.

Edit: Setting slippage of 0.25% means always hitting the worst case and I honestly don't know why I used it, you may want to experiment with it, either set to 0, make it dependent upon ATR, or randomize.

Edit 2: generation of stop loss signals is now correct.

[jacopomaroli]

In my current strategy I'm basing my RSI on a window of 14 samples of 1 minute so that's actually aligned with my current BOT implementation. Sorry I should have mentioned that. I wasn't expecting the same RSI graph of a 14 days window :)

I put the fees to 0.00075 which should align with my broker fees of 0.075% and put the slippage to 0 to have a fair comparison with the other backtesting suite but still got negative results.

https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/ba493f600d6581bd711ac5cfcd690ceb/raw/cbf385be789b7b356a14b021a73350d83ae9806b/strategy-cmp.ipynb

Start                     2021-03-04 00:00:00+00:00
End                       2021-03-10 23:58:00+00:00
Duration                            6 days 06:31:00
Init. Cash                                      100
Total Profit                               -26.5796
Total Return [%]                           -26.5796
Benchmark Return [%]                        10.8492
Position Coverage [%]                       46.5397
Max. Drawdown [%]                           27.1941
Avg. Drawdown [%]                           6.94754
Max. Drawdown Duration              6 days 04:31:00
Avg. Drawdown Duration              1 days 13:34:00
Num. Trades                                     160
Win Rate [%]                                  48.75
Best Trade [%]                             0.578857
Worst Trade [%]                             -3.6604
Avg. Trade [%]                            -0.182984
Max. Trade Duration                 0 days 02:11:00
Avg. Trade Duration          0 days 00:25:59.625000
Expectancy                                -0.160167
SQN                                         -3.5496
Gross Exposure                             0.465397
Sharpe Ratio                               -40.4191
Sortino Ratio                              -51.0524
Calmar Ratio                               -3.67727
dtype: object

I also tried to base the indicators on 'close' instead of 'open' because that's what the other suite is using but got similar results https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/9e029ba7bdee2179e854b17110548806/raw/7e46728d9c22f5fe6de7b39acf0ceac06a24c933/strategy-cmp-close.ipynb

Start                     2021-03-04 00:00:00+00:00
End                       2021-03-10 23:58:00+00:00
Duration                            6 days 06:31:00
Init. Cash                                      100
Total Profit                               -26.5796
Total Return [%]                           -26.5796
Benchmark Return [%]                        10.8492
Position Coverage [%]                       46.5397
Max. Drawdown [%]                           27.1941
Avg. Drawdown [%]                           6.94754
Max. Drawdown Duration              6 days 04:31:00
Avg. Drawdown Duration              1 days 13:34:00
Num. Trades                                     160
Win Rate [%]                                  48.75
Best Trade [%]                             0.578857
Worst Trade [%]                             -3.6604
Avg. Trade [%]                            -0.182984
Max. Trade Duration                 0 days 02:11:00
Avg. Trade Duration          0 days 00:25:59.625000
Expectancy                                -0.160167
SQN                                         -3.5496
Gross Exposure                             0.465397
Sharpe Ratio                               -40.4191
Sortino Ratio                              -51.0524
Calmar Ratio                               -3.67727
Name: Close, dtype: object

Also just for the sake of it I tried to remove the fees and got negative results as well

Start                     2021-03-04 00:00:00+00:00
End                       2021-03-10 23:58:00+00:00
Duration                            6 days 06:31:00
Init. Cash                                      100
Total Profit                                -6.5944
Total Return [%]                            -6.5944
Benchmark Return [%]                        10.8492
Position Coverage [%]                       46.5397
Max. Drawdown [%]                           9.40835
Avg. Drawdown [%]                             2.429
Max. Drawdown Duration              6 days 04:31:00
Avg. Drawdown Duration              1 days 13:23:15
Num. Trades                                     160
Win Rate [%]                                     65
Best Trade [%]                             0.729404
Worst Trade [%]                            -3.51304
Avg. Trade [%]                           -0.0330088
Max. Trade Duration                 0 days 02:11:00
Avg. Trade Duration          0 days 00:25:59.625000
Expectancy                               -0.0340817
SQN                                       -0.704412
Gross Exposure                             0.465397
Sharpe Ratio                               -8.79399
Sortino Ratio                              -11.4709
Calmar Ratio                               -10.4283
dtype: object

I haven't tried yet portfolio.orders.records_readable. I wanted to give you a quick feedback. I'll give it a go and report if I get any result in these days

[polakowo]

Could you share your backtrader code that produces the results you think make more sense? because looking at the stats alone I see a usual picture of a strategy that simply underperforms for the selected period of time. Without fees, you've got a decent win rate of 65%, but a negative expectancy gives you a hint that there are some outlier trades that skew your results into the negative zone.

If you zoom in into each of these trades, you can notice how the strategy repeatedly buys before a bigger price drop (coincidence or system?). If you run the same analysis on March 1st though, your strategy becomes slightly profitable.

Edit: notice that each RSI implementation is different, so you can also try the vbt.IndicatorFactory.from_talib('RSI') with output real, it gives slightly better results.

[jacopomaroli]

hey, thanks for getting back. I didn't have the time to go through all the feedbacks you gave me today. I did my best to put up this dirty mashed up notebook in a short time https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/28fc24475a0bab3fb72a7e46a7ffba57/raw/66f589d6ea12d9fdba2c8071b7710221c8fd2cbd/strategy-cmp.ipynb

It's not the cleanest code you'll see but that's basically what's going on with backtrader. I might take sometime tomorrow to clean it up and dive a bit deeper to compare it to the other indicators you mentioned and see if I can match them.

but yeah, I really wanted to get a preliminary implementation your hands today :)

edit: I'm thinking that I'm actually trading BTC/USDT and not BTC/USD so I might review that as well as it might have changed things slightly in the previous run. (I mean. not 20% much but still...)

edit2: updated notebook to fetch BTC/USDT data from binance edit3: updated to use ta-lib RSI and portfolio.orders.records_readable edit4: started to use ta-lib RSI for both vectorbt and backtrader and buy/sell events seems to align for the most part on a first glance

[polakowo]

RSI indicators differ only in the implementation of the moving average (see https://www.macroption.com/rsi-calculation/), in vectorbt it's SMA, and if ewm is set to True it uses the pandas CPython implementation with span set to the window size. Glad to see vectorbt and backtrader results being similar. I might want to create a simple notebook that shows porting a basic backtrader strategy to vectorbt.

Edit: There are some pitfalls, for example, backtrader executes an order at the next timestamp while in vectorbt you're responsible for shifting your signals to not execute the order at the current timestamp (I guess this can be controlled with cheat-on-close option). Also, vectorbt buys the maximum available size by pre-calculating fees, which I see you tried to emulate with a 99% sizer in bt.

Try to distill your example a bit, port it to vectorbt 0.17.2, and you're welcome to create a PR.

[jacopomaroli]

More updates! :) https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/01eb8687bb3d6c36addab02f83695769/raw/65d1dd39a472618959f7b84889312ad23cda977e/strategy-cmp.ipynb

I managed to take the resulting open/close positions from backtrader in post analysis, and feed them into vectorbt as raw buy/sell signals and overlapped them at the end of the notebook with a pure vectorbt/talib RSI strategy. It's interesting to notice how the lines are overlapping for the most part but slightly diverging at the end. I think this is what throws off the end result:

vectorbt

Total Profit                              -0.137513
Total Return [%]                          -0.137513

backtrader

Profit 5.595%
PnL Net  5.91 

I also matched the sizer with the commissions so now that's going to align as well. I'll try to mess around with the cheat-on-close and see if it gets better.

As for the PR it would be nice showing a strategy giving mediocre results with backtrader, tweaking it with vectorbt and show backtrader-backtesting results with overlapping graphs as I did, demonstrating the improved results. Maybe it's too much work though. Let's start to see if I can match the results :D

edit: mmmmh interesting. seems like the issue boils down to the strategy run with vectorbt not executing this transaction

54  54  2021-03-10 17:41:00+00:00   Close   0.001763    56431.18    0.074608    Buy
55  55  2021-03-10 18:42:00+00:00   Close   0.001763    56980.00    0.075334    Sell

edit2: and bingo. this is the comparison of the two RSI at 2021-03-10 17:41:00+00:00: vectorbt + talib

30.100519162546828

backtrader + talib

29.48790654036773

also activating cheat-on-close produced the same result

[polakowo]

Great, you're close :)

That's interesting, any ideas why RSI values differ? they both seem to run the same C library, which means input to the RSI is different (either different price or params passed, or any hidden flags set by bt?). I would also extract signals from the bt strategy and run assert equality with that of vectorbt before comparing their performance.

Edit: Regarding PR, comparing a basic RSI strategy using vectorbt and bt is already useful for many users. Especially extracting meta such as signals and orders from bt and comparing them with that of vectorbt to ensure that both packages are "talking" the same language and producing the same artifacts. Debugging is a very powerful tool, and your notebook shows how to compare two different implementations of the same strategy to find potential issues, which is already solid.

Improving the strategy on paper is rather straightforward, where you could build a grid of parameters (incl. frequency, window size, types of rolling windows, etc.) and search for the best fit, although this would introduce new challenges such as overfitting etc. + there is no easy way to tell whether the strategy has been really improved or you just cherry-picked noise.

[wukan1986]

EMA and RSI, have two mode: METASTOCK/Metastock, you can set by ta.set_compatibility(1)

https://github.com/TA-Lib/ta-lib/blob/master/src/ta_func/ta_RSI.c#L110 https://github.com/TA-Lib/ta-lib/blob/master/src/ta_func/ta_EMA.c#L291

https://github.com/mrjbq7/ta-lib/issues/244

[jacopomaroli]

hey wukan, thanks for the heads up, I gave a quick check in both vectorbt and backtrader repo and couldn't find set_compatibility mentioned anywhere in the code so I assume they'd both use DEFAULT rather than METASTOCK.

https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/c66cc646e6fc8a99d2f56912d524eaad/raw/7971221bd3201429919fa207dc942fc6dbcafa5c/strategy-cmp.ipynb

I updated the notebook to extrapolate what I believe being the rolling indicators data after the backtrader strategy run (I'll doublecheck but I'm 90% sure is not the last window)

I found out you need to shift backtrader's RSI signal points ahead by one row respect to the timestamp in order to align with vectorbt. I then plotted the delta between vectorbt RSI and backtrader RSI which seems to settle on a +/- 1 with occasional -2 spikes. also there's a big +8 spike at the beginning. (they seems to handle differently the initial situation when there's not enough point to have a real RSI)

In order to use talib with vectorbt I had to compile it myself in the notebook, instead the backtrader one came with the python package IIRC. On top of my head I can only think about weird compilation flags about double precision and stuff like that, but I find it difficoult to justify +/- 2 on the RSI.

maybe there's a way to use the system ta-lib rather than the one which comes prebundled with backtrader. (if manage to confirm there's inded a precompiled talib shipped with backtrader)

Despite this, I guess the whole RSI-gate doesn't explain why if I extrapolate the buy/sell events from backtrader and put them in vectorbt I get a different report

Backtrader: PnL Net 11.58 
vs
VectorBT: Total Profit 1.93099

maybe I'll try and shift ahead the buy/sell signals as I did for the RSI. I guess it has a lot to do with what you said in one of your previous comments

backtrader executes an order at the next timestamp while in vectorbt you're responsible for shifting your signals to not execute the order at the current timestamp

edit: shifting ahead backtrader sell/buy signal before feeding them to vectorbt yield to the following:

Total Profit: 0.965896

we're still far from 11.58 https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/134110495f0b3de71129f8d42c4e0fbf/raw/12af6279223fb591ba925f6d7b8fd09a4997a57e/strategy-cmp.ipynb

[polakowo]

I played with your notebook a bit. If you set fees to zero in bt and vbt and feed the bt transactions directly into vbt, they will produce the same results:

print(bt_transactions)
                       amount     price  sid symbol       value
2021-03-04 02:36:00  0.001952  51224.09    0        -100.000000
2021-03-04 06:10:00 -0.001952  49867.38    0          97.351422
2021-03-04 06:23:00  0.001975  49285.77    0         -97.351422
2021-03-04 08:02:00 -0.001975  50021.32    0          98.804313
2021-03-04 08:53:00  0.001970  50151.71    0         -98.804313
...                       ...       ...  ...    ...         ...
2021-03-10 17:37:00  0.001964  56631.53    0        -111.200283
2021-03-10 18:42:00 -0.001964  57057.06    0         112.035843
2021-03-10 19:43:00  0.001972  56817.36    0        -112.035843
2021-03-10 21:31:00 -0.001972  56526.06    0         111.461440
2021-03-10 23:03:00  0.001969  56614.45    0        -111.461440

[79 rows x 5 columns]

print(vbt.Portfolio.from_orders(bt_transactions['price'], bt_transactions['amount'], fees=0.).value())
# or print(vbt.Portfolio.from_signals(bt_transactions['price'], bt_transactions['amount'] > 0, bt_transactions['amount'] < 0, fees=0.).value()
2021-03-04 02:36:00    100.000000
2021-03-04 06:10:00     97.351422
2021-03-04 06:23:00     97.351422
2021-03-04 08:02:00     98.804313
2021-03-04 08:53:00     98.804313
                          ...    
2021-03-10 17:37:00    111.200283
2021-03-10 18:42:00    112.035843
2021-03-10 19:43:00    112.035843
2021-03-10 21:31:00    111.461440
2021-03-10 23:03:00    111.461440
Length: 79, dtype: float64

As suggested, setting execution price to the shifted close will produce the same results as reported by bt:

# bt
Final Portfolio Value: 110.28

# vbt
bt_portfolio = vbt.Portfolio.from_signals(ohlcv['Close'], bt_entries, bt_exits, price=ohlcv['Close'].vbt.fshift(1), fees=0)
bt_portfolio.value()
start
2021-03-04 00:00:00+00:00    100.000000
2021-03-04 00:01:00+00:00    100.000000
2021-03-04 00:02:00+00:00    100.000000
2021-03-04 00:03:00+00:00    100.000000
2021-03-04 00:04:00+00:00    100.000000
                                ...    
2021-03-10 23:56:00+00:00    110.126174
2021-03-10 23:57:00+00:00    110.186478
2021-03-10 23:58:00+00:00    110.281530
2021-03-10 23:59:00+00:00    109.959536
2021-03-11 00:00:00+00:00    110.282416
Name: Close, Length: 9991, dtype: float64

This is because bt executes at the next bar, while vectorbt executes based solely on the vector you provided it with, which is the current close in your case.

Edit: then you need to go a step further and add a commission of 0.075% and a fixed sizer of say 0.0001 BTC (instead of a variable one that is harder to debug). If you're able to extract how much fees you paid for each transaction in bt, you can then compare costs. In vbt, having a price of 51224.09 will produce 51224.09 * 0.0001 * 0.00075 = 0.00384181 fees that are deducted from your cash account.

[jacopomaroli]

Looks like I made it in the end! https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/3d2df4af6f4db45ce8955079cf630839/raw/8cd7ac2523b75911114095c867f9ee0c04656979/strategy-cmp.ipynb

There was a bug in the original project on how it was setting up fees (passed totally wrong params to BT's setcommission function causing commission to be always 0). After I fixed that it started to show coherent results

weirdly enough setting up a sizer in vectorbt didn't work. in the end just setting the commissions in the initial configuration was enough to get results to match. I added a sizer with

rsi_portfolio = vbt.Portfolio.from_signals(ohlcv['Close'], rsi_entries, rsi_exits, price=ohlcv['Close'].vbt.fshift(1), size=(100 - fees)/100, size_type='percent')

but got slightly different results. removing size/size_type caused results to align back again. (Maybe there's another bug in the way the sizer was set originally in bt and is not really doing anything?)

In the end I fed BT's RSI (from the original backtesting result) into vectorbt and generated entries/exits and then proceded as usual. also I compared BT/VBT entries and exits between each other on a graph with an xor and proved they matched (apart from vectorbt setting up a sell as a first action which doesn't change the result anyway. If there's a way to iron this out, please let me know) I know I don't need to reset buy/sells but is just for cosmetic purposes and show they overlap with bt ones.

If you want to have one last overall look just to confirm I'm not doing anything stupid I'll proceed with further cleanup and writeup and I'll open a PR. Showing the difference in results depending on the RSI implementation you use could be a great point to make as a bonus addendum. I would have neve though an indicator implementation could cause that much difference in the final outcome!

[polakowo]

I will create a convenience method in 0.17.3 to clean up entry and exit signals if you could wait a day or two.

I'm not quite sure why sizer in vectorbt didn't work for you. Also, setting both the fees and a percentage sizer to (100 - fees)/100 is different from just setting the fees: to spend $100 you order (1 - 0.00075) * 100 = 99.925 value of shares and then apply fees of 99.925 * 0.00075 = 0.07494375, both summing to 99.99994375, leaving you with a little cash leftover. I'm not quite sure how bt calculates fees, but if you try to order the maximum number of shares with vbt, it calculates how many shares can be ordered such that, with fees applied, their value exactly matches the amount of cash you have. For example, to spend $100, it orders 100 / (1 + 0.00075) = 99.92505620784411 value of shares, which if you apply fees on, gives you exactly $100, such that your cash balance becomes zero after this transaction. That why when doing a comparison with other libs, you should strive to compare their transactions by value and paid fees to be sure that they are effectively producing the same result (In your case, at least comparing bt_transactions with vbt_portfolio.order_records).

Here are some suggestions for your notebook:

• Instead of or additionally to building a vbt.Portfolio from bt signals, you can build it from bt_transactions directly using vbt.Portfolio.from_orders
• There is no need to call vbt.plotting.Scatter(data=your_df), you could just do your_df.vbt.plot() or when plotting signals your_df.vbt.signals.plot().
• Holding portfolio can be created with a one-liner vbt.Portfolio.from_holding
• Since you're using IndicatorFactory to build RSI, you can do indexing on the indicator instance itself (instead of on rsi.real) but also extract crossover signals using rsi.real_above and rsi.real_below with crossover flags.
• Timezone management all over the place adds complexity, consider doing timezone localizations and conversions at a single place (closer to the data definition) and then reusing this index.
• Maybe consider using vbt.BinanceData, it uses python-binance and is already timezone-aware, but it would require credentials.

Edit: I'm porting ccxt to vectorbt right now, will release in 0.17.3, so you could use vbt.CCXTData soon.

[jacopomaroli]

Quick update: https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/3c50dd8e37e000473d0adf9ea34d249c/raw/ae0a8737acd1978970b1ecdc369c0ea28568097d/strategy-cmp.ipynb

I'm not done but since I've been working on it for a while I wanted to post an update just so you know I didn't give up :)

I couldn't find a way to extrapolate the fees from the results through an analizer in backtrader so I just put them in an array and retrieved them from the result anyway. I then plotted the difference on a graph which seems to be very small when not setting a sizer in vbt so maybe the way vbt is trying to maximize the buy matches very closely backtrader's sizer + commission.

regarding your suggestions in order:

• I think I'll show both approches as I believe there's value in showing how to create a portfolio from signals so you can change params much faster in vbt and be sure you'll match the numbers when feeding them back in backtrader notebook is now showing both approaches
• I tried but it didn't show anything. I'll make sure to post a version of the notebook which highlights the problem
• cool I'll give it a try now it's using vbt.Portfolio.from_holding
• I left it open so the reader can do more complex implementations such as

  (rsi_df.rsi < rsi_bottom) & ((stoch.slowd < stoch_slowd_bottom) | (stoch.slowk < stoch_slowk_bottom)) & (macd_df.macdhist > macd_hist_top)

  ~~which seems much easier to port back and forth from backtrader. I wasn't sure how to obtain the same flexibility when using rsi.real_above and rsi.real_below. Anyway it looks like it might be stretching the scope too much so I'll go for your suggestion.~~ done

• I think I managed to move most or all of those where I extract results from backtrader and deal with zone-aware dates for the rest of the notebook done
• I switched to BinanceData. luckily there was no need to use credentials

As I'm not fetching anymore data with my approach there should be no need for me use CCXTData but that's a cool addition for the future!

edit: updated list with progresses

[polakowo]

Looks good. Regarding plots:

vbt.plotting.Scatter(
    data=vbt_entries ^ bt_entries,
    trace_names=['Entries (Delta)'],
    x_labels=bt_entries.index
).fig.show_png()

can be changed to

(vbt_entries ^ bt_entries).rename('Entries (Delta)').vbt.signals.plot().show_png()

and

vbt.plotting.Scatter(
    data=pd.concat([rsi_vbt_tf, rsi_bt_tf], axis=1),
    trace_names=['RSI (VBT)', 'RSI (BT)'],
    x_labels=rsi_bt_tf.index
).fig.show_png()

can be changed to

pd.DataFrame({'RSI (VBT)': rsi_vbt_tf['rsi'], 'RSI (BT)': rsi_bt_tf['rsi']}).vbt.plot().show_png()

When plotting signals, always use vbt.signals.plot because sometimes using Scatter alone flips the y axis (plotly bug).

I'm releasing 0.17.3 which includes a convenient method to clean up signals.

vbt_bt_entries = rsi_entries.vbt.signals.first(reset_by=rsi_exits, allow_gaps=True)
vbt_bt_exits = rsi_exits.vbt.signals.first(reset_by=rsi_entries, allow_gaps=True)

can be changed to

vbt_bt_entries, vbt_bt_exits = pd.DataFrame.vbt.signals.clean(vbt_bt_entries, vbt_bt_exits)
# or vbt_bt_entries, vbt_bt_exits = vbt_bt_entries.vbt.signals.clean(vbt_bt_exits)

It also by default removes any exit signal coming before the first entry signal.

Edit: cheat-on-close option can be removed as you're already shifting data like bt.

[jacopomaroli]

https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/bc50449b8d123b58030a249f2f606e7f/raw/9d662942d5e9c482afd23064cbaf8b17ca3d4cc1/strategy-cmp.ipynb

done and done! I believe there's everything now.

I think something broke with BinanceData as download output is

2021-03-04 00:00:00+00:00 - 2021-03-04 08:19:00+00:00: : 1it [00:02,  2.38s/it]

vs CCXTData

2021-03-04 00:00:00+00:00 - 2021-03-10 23:59:00+00:00: : 21it [00:11,  1.86it/s]

and is fetching way less stuff.

Let me know if you're happy with it code-wise so I'll start with the writeup!

side note: I saw you released 1w ago sklearn support https://github.com/polakowo/vectorbt/commit/ec0178f0fe4276e5f823ead4726b6f90269a1977 cool stuff, that's gonna be fun! :D

[polakowo]

Oh right, kudos for finding the issue with BinanceData, fix is on the way!

The notebook looks great code-wise.

Yep, I added lots of features for convenience recently (there is even a telegram bot in the new version), just keep burning my time :)

[jacopomaroli]

https://nbviewer.jupyter.org/urls/gist.githubusercontent.com/jacopomaroli/ae67b95672567d15bff343c5a6906791/raw/2b3f1118f6ce5e51fe49ebef64f00fe4ecd3e72b/strategy-backtrader-vectorbt.ipynb I think it's ready! Let me know if you're happy for me to open a PR :)

[polakowo]

Yep, you can open a PR. You would need to add your notebook to the examples folder and post the nbviewer link both to readme and __init__.py (which will appear in the docs). Just for the sake of consistency, use a camel-cased name for the notebook like "PortingBTStrategy.ipynb" and I suggest describing it as "Porting a backtrader strategy".

I'm currently working on a similar notebook that shows how to do pairs trading with both bt and vbt (percentage sizer, two data feeds, etc.), such that both notebooks can cover a good amount of stuff for bt users.

[jacopomaroli]

Done, I also added it as a title as first section inside the notebook itself :)

Awesome, showing how to port strategies will definitely make the project even more attractive for new users! As for me your addition of sklearn support inspired me in giving it a shot to a genetic algorithm approach for indicators parameters tuning

[polakowo]

True, although new users must already be attracted by the fact that vectorbt is so much easier to use. It amazes me how much boilerplate code must be written to define a simple bt strategy, whereas it's a couple of lines in vbt. But it would be unfair comparing both libs as they are working differently, and I usually do a final backtest with bt anyway. Some parts of vbt are still beginner unfriendly (such as vbt.Portfolio.from_order_func), but I guess this is the price you pay for maximum flexibility and performance.

The sklearn support I added is primarily around splitting, scaling, and transforming functions (so you can conveniently do df.vbt.zscore(), for example), rather than models, which may be added in the future. Genetic and also RL algos are worth trying for hyperparameter optimization, but I'm very skeptical towards complex algorithms trying to deal with noisy time series data as they tend to overfit quickly. But who knows, maybe you're lucky to find a gem :)

[AliSahlolbei]

Hello, thanks for this library, I was trying to use the built in vbt.RSI but the output is completely wrong, although when setting ewm to True it looks similar but the numbers doesn't match talib or tradingview output

[polakowo]

@AliSahlolbei please see this reply.

[AliSahlolbei]

Thanks