Backtest results changed after version upgrade

SeifallahSnoussi commented 2 years ago

Describe the bug I had different backtest results after I've upgraded Autotrader from 0.6.5 to 0.6.6.

In the backtest in version 0.6.6:

Only the first position was opened and all the signals, coming after that, were canceled.
The first position has been stayed open until the end
NAV passed immediately to $-208923.93 after I opened first Position (see screenshot below)

In the backtest in version 0.6.5:

Everything was OK

Backtest results in version 0.6.5

Start date:              Jan 05 2021 16:00:00
End date:                Jun 03 2022 20:00:00
Starting balance:        $32000.0
Ending balance:          $39126.36
Ending NAV:              $39127.56
Total return:            $7126.36 (22.3%)
Total no. trades:        99
Total fees:              $194.136
Backtest win rate:       44.4%
Maximum drawdown:        -11.66%
Max win:                 $1940.71
Average win:             $526.24
Max loss:                -$2181.52
Average loss:            -$291.42
Longest win streak:      5 trades
Longest losing streak:   8 trades
Trades still open:       1
Cancelled orders:        59

Backtest results in version 0.6.6

Start date:              Jan 05 2021 16:00:00
End date:                Jun 03 2022 20:00:00
Duration:                514 days 04:00:00
Starting balance:        $32000
Ending balance:          $32000
Ending NAV:              $-208923.93
Total return:            $0 (0.0%)

 No trades closed.
Trades still open:       1
Cancelled orders:        158

My strategy params

NAME: 'SMA2 Strategy'
MODULE: 'SMA2'
CLASS: 'SMA2'
INTERVAL: 'H1'
PERIOD: 40
RISK_PC: 100
SIZING: 'risk'
PARAMETERS:

  # Exit level parameters
  RR: 6
WATCHLIST: ['XAG_USD',]

Strategy signals

to initiate a Long position I set this code:
```
 new_order = Order(direction=1)
```
For Short position I set this code:
```
new_order = Order(direction=-1)
```
Take profit or Stop Loss when it reaches 6%

kieran-mackle commented 2 years ago

I believe this is due to a change in how the bid/ask spread is included in backtests by the virtual broker. In v0.6.5 and below, the spread argument in the backtest method was treated as a spread value in pips, where 1 pip = 0.0001 price units. Of course, this is not true for all instruments, and so in v0.6.6, I changed the definition of spread to refer to the bid/ask spread in price units. So, if you previously had spread=0.5, you should update this to spread=0.00005 for equivalent results.

Apologies for not having a clear warning on this change, but ultimately I think it is a better approach to including the spread cost in backtests.

Please let me know if this doesn't fix the issue you are seeing.

SeifallahSnoussi commented 2 years ago

Thanks. that was it, I forgot to mention that I've configured spread cost in backtests.

I've changed the value as you asked me, and I got the same exact results as it was in the previous version (0.6.5)

kieran-mackle commented 2 years ago

Great, glad it was a simple fix!

kieran-mackle / AutoTrader