TypeError: Index.get_loc() got an unexpected keyword argument 'method'

luludocteur commented 3 months ago

Hi everybody,

I'm kind of a new user of backtesting.py. I've coded a litlle strategy of grid trading of BTC in 1 minute. I make use of the ATR indicator used for Stop Loss that i resampled in daily. My problem is linked to the plot() method. Indeed my code is working well because i can print the Statistics with stat = bt.run().

If you have any questions, feel free to ask, i'll be pleased to answer you. Thank you

Expected Behavior

Running the strategy and printing the plot

Actual Behavior

When the code launch the plot() method, it launch the _plotting.py program. I let you check the whole error but the last problem is the exact title of this topic : "TypeError: Index.get_loc() got an unexpected keyword argument 'method'". I digged a little into the code and it seems that this argument is expected in the _group_trades function:

  def _group_trades(column):
    def f(s, new_index=pd.Index(df.index.view(int)), bars=trades[column]):
        if s.size:
            # Via int64 because on pandas recently broken datetime
            mean_time = int(bars.loc[s.index].view(int).mean())
            new_bar_idx = new_index.get_loc(mean_time, method='nearest')
            return new_bar_idx
    return f

The whole traceback:

/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:122: UserWarning: Data contains too many candlesticks to plot; downsampling to '5T'. See `Backtest.plot(resample=...)`
  warnings.warn(f"Data contains too many candlesticks to plot; downsampling to {freq!r}. "
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:126: FutureWarning: 'T' is deprecated and will be removed in a future version, please use 'min' instead.
  df = df.resample(freq, label='right').agg(OHLCV_AGG).dropna()
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:128: FutureWarning: 'T' is deprecated and will be removed in a future version, please use 'min' instead.
  indicators = [_Indicator(i.df.resample(freq, label='right').mean()
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:136: FutureWarning: 'T' is deprecated and will be removed in a future version, please use 'min' instead.
  equity_data = equity_data.resample(freq, label='right').agg(_EQUITY_AGG).dropna(how='all')
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:153: FutureWarning: 'T' is deprecated and will be removed in a future version, please use 'min' instead.
  trades = trades.assign(count=1).resample(freq, on='ExitTime', label='right').agg(dict(
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:147: FutureWarning: Series.view is deprecated and will be removed in a future version. Use ``astype`` as an alternative to change the dtype.
  mean_time = int(bars.loc[s.index].view(int).mean())
Traceback (most recent call last):
  File "/Users/alexanderlunel/Documents/Python/Bots/grid_bot/btmodele.py", line 99, in <module>
    bt.plot()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/backtesting.py", line 1592, in plot
    return plot(
           ^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py", line 203, in plot
    df, indicators, equity_data, trades = _maybe_resample_data(
                                          ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py", line 153, in _maybe_resample_data
    trades = trades.assign(count=1).resample(freq, on='ExitTime', label='right').agg(dict(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/resample.py", line 352, in aggregate
    result = ResamplerWindowApply(self, func, args=args, kwargs=kwargs).agg()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/apply.py", line 190, in agg
    return self.agg_dict_like()
           ^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/apply.py", line 423, in agg_dict_like
    return self.agg_or_apply_dict_like(op_name="agg")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/apply.py", line 1608, in agg_or_apply_dict_like
    result_index, result_data = self.compute_dict_like(
                                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/apply.py", line 497, in compute_dict_like
    getattr(obj._gotitem(key, ndim=1), op_name)(how, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 294, in aggregate
    return self._python_agg_general(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 327, in _python_agg_general
    result = self._grouper.agg_series(obj, f)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/ops.py", line 864, in agg_series
    result = self._aggregate_series_pure_python(obj, func)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/ops.py", line 885, in _aggregate_series_pure_python
    res = func(group)
          ^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 324, in <lambda>
    f = lambda x: func(x, *args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py", line 148, in f
    new_bar_idx = new_index.get_loc(mean_time, method='nearest')
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Index.get_loc() got an unexpected keyword argument 'method'

Steps to Reproduce


import numpy as np 
import pandas as pd
import pandas_ta as ta
from backtesting import Strategy, Backtest
from backtesting.lib import resample_apply

df = pd.read_csv('/Users/alexanderlunel/Documents/Python/Bots/grid_bot/data.csv', sep=';')
df['Open_time'] = pd.to_datetime(df['Open_time'], unit='ms')
df.set_index(df['Open_time'], inplace = True)

#Grille
def grid(midprice, space, n):
    return np.arange(midprice-(n/2*space), midprice+(n/2*space), space)

midprice, space, n = 20000, 200, 10
grid = grid(midprice, space, n)

signal = [0]*len(df)

i = 0
for date, candle in df.iterrows():
    for p in grid:
        if p>candle.Low and p<candle.High:
            signal[i]=p
    i+=1

df['signal']=signal

def SIGNAL():
    return df.signal

class Strat(Strategy):

    nb_short = 0
    nb_long = 0
    size = 2
    maxdrawdown = 0.1
    grid = grid
    space = 200

    def init(self):
        self.signal = self.I(SIGNAL)
        self.atr = resample_apply('1D', ta.atr, self.data.High.s, self.data.Low.s, self.data.Close.s, length = 14)
        #self.atr = self.I(ta.atr, self.data.High.s, self.data.Low.s, self.data.Close.s, length = 14)
        #self.ratio = 0.05
        # daily_close = self.data.df.resample('D', label='right', on='Open_time')['Close'].agg('last')
        # daily_low = self.data.df.resample('D', label='right', on='Open_time')['Low'].agg('last')
        # daily_high = self.data.df.resample('D', label='right', on='Open_time')['High'].agg('last')
        # self.atr = self.I(ta.atr, daily_high, daily_low, daily_close, 14)

    def next(self):
        before = self.data.Close[-2]
        price = self.data.Close[-1]

        if (self.signal[-1] in self.grid):
            if len(self.trades) != 0 and self.signal[-1] != self.trades[-1].entry_price:
                if (self.nb_long_short()[0] <= 3) and (before > price):
                        slu = price - self.atr[-1]
                        tpu = price + self.space
                        self.buy(tp=tpu, sl=slu, size=self.size)
                        self.nb_long+=1
                elif (self.nb_long_short()[1] <= 3) and (before > price):
                        sld = price + self.atr[-1]
                        tpd = price - self.space
                        self.sell(tp=tpd, sl=slu,size=self.size)
                        self.nb_short+=1
            else:
                if before > price:
                    slu = price - self.atr[-1]
                    tpu = price + self.space
                    self.buy(tp=tpu, sl=sld, size=self.size)
                    self.nb_long+=1
                elif before < price:
                    sld = price + self.atr[-1]
                    tpd = price - self.space
                    self.sell(tp=tpd, sl=sld, size=self.size)
                    self.nb_short+=1

    def nb_long_short(self):
        long, short = 0, 0
        for trade in self.trades:
            if trade.is_long:
                long+=1
            elif trade.is_short:
                short+=1
        self.nb_long = long
        self.nb_short = short
        return self.nb_long, self.nb_short

bt = Backtest(df, Strat, cash=10_000_000, hedging = True, margin=1/10, commission=0.000, exclusive_orders=False)

stat = bt.run()
print(stat)
bt.plot()

# stats, heatmap = bt.optimize(
#     TPSLratio = list(np.arange(0.1,2,100)),
#     maximize = 'Return [%]',
#     max_tries = 200,
#     return_heatmap=True
# )[25000. 25350. 25700. 26050. 26400. 26750. 27100. 27450. 27800. 28150.]

Additional info

Backtesting version: 0.3.3
bokeh.__version__:
OS: MAC OS Monterey Version 12.2.1

devansh-alphagrep commented 3 months ago

Same issue. seems like the API has changed in pandas 2.0.3

luludocteur commented 3 months ago

Yes i agree, probably a version issue. But the fact is that it work perfectly fine when i tried to plot simple strategies available on this github. So i think it also have something to do with resampling the indicator. idk..

If you success to fix it, i'm curious

devansh-alphagrep commented 3 months ago

@luludocteur the pypy repo hasn't been updated in a while, so it still uses the old pandas.

Just clone this repo from github and run pip install -e . in the repo to install the latest backtesting.py

Joyen12 commented 1 month ago

This is still an ongoing problem. Why don't you just update the repo ?

Best regards

vasinl124 commented 1 month ago

I just put this in requirements.txt

backtesting @ git+https://github.com/kernc/backtesting.py@master

Joyen12 commented 1 month ago

I just put this in requirements.txt

backtesting @ git+https://github.com/kernc/backtesting.py@master

Yeah I fixed it too directly in site-packages lol but that would still be better for future user to have something working right away without them to have to search why it's not working.

vasinl124 commented 1 month ago

the last update was in 2021 soooooooo little chance it will happen, I don't know :(

oliver-zehentleitner commented 3 weeks ago

Here is a fork in which the error does not occur. It was solved by locking old versions in the dependencies. The aim was to create a stable version. We will subsequently migrate to newer versions.

https://github.com/LUCIT-Systems-and-Development/lucit-backtesting

kernc / backtesting.py