[BUG] custom strat always has short positions even though it should not

zonemikel commented 3 years ago

Problem description

I wrote my own custom strat, I basically just wanted MACD with stop loss and take profit. I know there is already a stop loss condition but I was just implementing it as a test. The built in stop loss works well with the macd by the way. Anyway no matter what I do it always has a 'short' position when I backtest my strat, even though I never return 'True' in the sell_signal. So basically there is one too many sells and I don't know where the other one comes from, so this causes a short position to come in and ruin the returns. This could ofc be in my code somewhere, but I started with the original macd and added to it. Even when I put in the original return in 'sell_signal' I still get this really weird sell/short.

MyStrat

from __future__ import (
    absolute_import,
    division,
    print_function,
    unicode_literals,
)

# Import modules
import backtrader as bt

# Import from package
from fastquant.strategies.base import BaseStrategy

class MACD2(BaseStrategy):
    """
    Moving Average Convergence Divergence (MACD) strategy
    Simple implementation of backtrader MACD reference: https://www.backtrader.com/blog/posts/2016-07-30-macd-settings/macd-settings/
    Summary:
    Enter if the macd line crosses the signal line to the upside and a control Simple Moving Average has had a
    net negative direction in the last x periods (current SMA value below the value x periods ago).
    In the opposite situation, given a market position exists, a sell position is made.
    Parameters
    ----------
    fast_period : int
        The period used for the fast exponential moving average line (should be smaller than `slow_upper`)
    slow_period : int
        The period used for the slow exponential moving average line (should be larger than `fast_upper`)
    signal_period : int
        The period used for the signal line for MACD
    sma_period : int
        Period for the moving average (default: 30)
    dir_period: int
        Period for SMA direction calculation (default: 10)
    """

    params = (
        ("fast_period", 12),  # period for the fast moving average
        ("slow_period", 26),
        ("signal_period", 9),
        ("sma_period", 30),
        ("dir_period", 10),
        ("take_profit",999),
        ("stop_loss",999)
    )

    def __init__(self):
        # Initialize global variables
        super().__init__()
        # Strategy level variables
        self.fast_period = self.params.fast_period
        self.slow_period = self.params.slow_period
        self.signal_period = self.params.signal_period
        self.sma_period = self.params.sma_period
        self.dir_period = self.params.dir_period
        self.last_buy = 0
        # setup stop loss and take profit stuff a %, like .5 = 50% 
        self.take_profit = self.params.take_profit
        self.stop_loss = self.params.stop_loss

        if self.strategy_logging:
            print("===Strategy level arguments===")
            print("fast_period :", self.fast_period)
            print("slow_period :", self.slow_period)
            print("signal_period :", self.signal_period)
            print("sma_period :", self.sma_period)
            print("dir_period :", self.dir_period)
        macd_ind = bt.ind.MACD(
            period_me1=self.fast_period,
            period_me2=self.slow_period,
            period_signal=self.signal_period,
        )
        self.macd = macd_ind.macd
        self.signal = macd_ind.signal
        self.crossover = bt.ind.CrossOver(
            self.macd, self.signal
        )  # crossover buy signal

        # Control market trend
        self.sma = bt.indicators.SMA(period=self.sma_period)
        self.smadir = self.sma - self.sma(-self.dir_period)

    def buy_signal(self):
        # Buy if the macd line cross the signal line to the upside 
        # and a control Simple Moving Average  has had a net negative 
        # direction in the last x periods 

        #return self.crossover > 0 and self.smadir < 0.0
        if self.last_buy != 0: # if we are long, dont take a second buy pos
            return False
        buy = self.crossover > 0 and self.smadir > 0.0       # MACD crosses signal line upward
        if buy and self.last_buy == 0: # if we are going to buy, and we aren't already long
            self.last_buy = self.dataclose[0] # store the last buy we had
            self.tp = self.dataclose[0] + self.dataclose[0]*self.take_profit
            self.sl = self.dataclose[0] - self.dataclose[0]*self.stop_loss
            print('Bought: %.2f TP: %.2f SL: %.2f'%(self.last_buy, self.tp, self.sl))
        return buy 

    def sell_signal(self):
        sell = self.last_buy != 0 and (self.dataclose[0] > self.tp or self.dataclose[0] < self.sl)
        if sell:
            if self.dataclose[0] > self.tp:
                print('sell tp: ', self.dataclose[0]-self.last_buy)
            else:
                print('sell sl: ', self.dataclose[0]-self.last_buy)
            self.last_buy = 0
        # allow for base macd
        '''if not sell:
            sell = self.crossover < 0 and self.smadir > 0.0
        if sell:
            print('Sell Macd Trigger: ',self.dataclose[0])
            self.last_buy = 0'''
        return sell

dft = get_stock_data("TSLA", "2000-01-01", "2021-05-25")
smdf = dft.iloc[-500:]
result, history = backtest(MACD2, smdf, verbose=False, plot=True, commission=0.0, return_history=True, take_profit=.1, stop_loss=.1)

Bought: 48.94 TP: 53.83 SL: 44.04 sell tp: 10.998001098632812 Bought: 71.68 TP: 78.85 SL: 64.51 sell tp: 9.129997253417969 Bought: 176.31 TP: 193.94 SL: 158.68 sell tp: 28.697998046875 Bought: 223.93 TP: 246.32 SL: 201.53 sell tp: 50.3900146484375 Bought: 367.13 TP: 403.84 SL: 330.42 sell tp: 42.868011474609375 Bought: 446.65 TP: 491.31 SL: 401.98 sell sl: -58.6099853515625 Bought: 441.61 TP: 485.77 SL: 397.45 sell tp: 45.030029296875 Bought: 655.90 TP: 721.49 SL: 590.31 sell tp: 73.8699951171875

You can see below in bold there are two sells in a row, and there is always a sell there, it makes no sense to me. strat_id strat_name dt type price size value commission pnl 0 0 stop_loss0.1_take_profit0.1 2019-10-02 buy 48.938000 2041 99882.457439 0.0 0.000000 1 0 stop_loss0.1_take_profit0.1 2019-10-25 sell 65.625999 -2041 99882.457439 0.0 34060.207439 2 0 stop_loss0.1_take_profit0.1 2019-12-16 buy 71.678001 1868 133894.506622 0.0 0.000000 3 0 stop_loss0.1_take_profit0.1 2019-12-20 sell 81.117996 -1868 133894.506622 0.0 17633.910309 4 0 stop_loss0.1_take_profit0.1 2020-06-03 buy 176.311996 859 151452.004959 0.0 0.000000 5 0 stop_loss0.1_take_profit0.1 2020-06-11 sell 194.567993 -859 151452.004959 0.0 15681.901169 6 0 stop_loss0.1_take_profit0.1 2020-07-02 buy 223.925995 746 167048.792175 0.0 0.000000 7 0 stop_loss0.1_take_profit0.1 2020-07-07 sell 277.971985 -746 167048.792175 0.0 40318.308533 8 0 stop_loss0.1_take_profit0.1 2020-08-18 buy 367.127991 565 207427.314758 0.0 0.000000 9 0 stop_loss0.1_take_profit0.1 2020-08-24 sell 402.839996 -565 207427.314758 0.0 20177.283173 10 0 stop_loss0.1_take_profit0.1 2020-09-08 sell 330.415192 -565 -186684.583282 0.0 0.000000 11 0 stop_loss0.1_take_profit0.1 2020-10-14 buy 446.649994 927 -24997.285492 0.0 -65672.663269 12 0 stop_loss0.1_take_profit0.1 2020-10-30 sell 401.984995 -927 -65434.224106 0.0 -16168.729779 13 0 stop_loss0.1_take_profit0.1 2020-11-18 buy 441.609985 844 -103912.335983 0.0 -22388.119827 14 0 stop_loss0.1_take_profit0.1 2020-11-19 sell 499.269989 -279 123209.185913 0.0 16087.141022 15 0 stop_loss0.1_take_profit0.1 2020-12-18 buy 655.900024 212 139050.805176 0.0 0.000000 16 0 stop_loss0.1_take_profit0.1 2021-01-05 sell 735.109985 -212 139050.805176 0.0 16792.511719 17 0 stop_loss0.1_take_profit0.1 2021-03-05 sell 590.310022 -212 -125145.724658 0.0 0.000000

Expected behavior

I expect there will never be two sells in a row. I would point out I tried allow_short=False and it made no difference, because ofc that's the default

Environment

platform (e.g. Linux, OSX, Windows): Linux, jupyter notebook
fastquant version (e.g. 0.1.3.17): 0.1.4.0
installation method (e.g. pip, conda, source): PIP

Great project, thank you for your time.

enzoampil commented 3 years ago

@zonemikel Hello, yes this seems to be a bug with respect to our stop_loss feature, since it automatically assumes "shorting" is allowed (hence selling even when we don't hold the security).

We will have to fix take profit and stop loss support on BaseStrategy. In the meantime, I believe you can solve this by implementing your own "stop loss" support in your custom strategy (and don't use the stop_loss parameter in backtest for now) :smile:

enzoampil commented 3 years ago

Also @zonemikel , if you succeed with above, you might want to contribute the same to BaseStrategy with a PR if you want! :smile:

enzoampil commented 3 years ago

@zonemikel This should be fixed based on our latest merged PR #377 !

enzoampil / fastquant