Closed straj-granicy closed 2 years ago
Taking an educated guess, I'd say that you're missing populate_indicators
part (also mentioned in the tutorial) - or have a typo where the parameter name is spelled slightly differently.
Thank you, @xmatthias I fixed it. Now the issue is in your code here: `frames = [dataframe] for val in self.buy_ema_short.range: frames.append({ f'emashort{val}': ta.EMA(dataframe, timeperiod=val) })
merged_frame = pd.concat(frames, axis=1)`
Running this I've got the following error:
TypeError: cannot concatenate object of type '<class 'dict'>'; only Series and DataFrame objs are valid
Please advise how to fix it. I' m following your tutorial on https://www.freqtrade.io/en/stable/strategy-advanced/ and putting the code in populate indicators: ` # Calculate all tema_middle values
frames = [dataframe]
for val in self.buy_tema_middle.range:
frames.append({f'buy_tema_middle_{val}': ta.EMA(dataframe, timeperiod=val)})
# Append columns to existing dataframe
merged_frame = pd.concat(frames, axis=1)
# Calculate all tema_long values
frames = [dataframe]
for val in self.buy_tema_long.range:
frames.append({f'buy_tema_long_{val}': ta.EMA(dataframe, timeperiod=val)})
# Append columns to existing dataframe
merged_frame = pd.concat(frames, axis=1)`
Thank you in advance for your help.
The correct code should be the following (i've updated the documentation with this now):
frames = [dataframe]
for val in self.buy_ema_short.range:
frames.append(DataFrame({
f'ema_short_{val}': ta.EMA(dataframe, timeperiod=val)
}))
# Append columns to existing dataframe
merged_frame = pd.concat(frames, axis=1)
basically the DataFrame()
call was missing
Dear @xmatthias, Sorry for disturbing you again for this matter. I've done everything according to your guides, but I still have errors (don't know why, as I've checked everything several times)... Here's my code:
`from pandas import DataFrame from functools import reduce import pandas as pd
import talib.abstract as ta
from freqtrade.strategy import (BooleanParameter, CategoricalParameter, DecimalParameter, IStrategy, IntParameter) import freqtrade.vendor.qtpylib.indicators as qtpylib from technical.indicators import ichimoku, SSLChannels
class BBRSIADXCONFIG2V31(IStrategy):
INTERFACE_VERSION = 2
# Minimal ROI designed for the strategy.
# This attribute will be overridden if the config file contains "minimal_roi".
minimal_roi = {
"0": 0.069,
"6": 0.023,
"16": 0.005,
"39": 0
}
# Optimal stoploss designed for the strategy.
# This attribute will be overridden if the config file contains "stoploss".
stoploss = -0.237
# Trailing stoploss
trailing_stop = True
trailing_only_offset_is_reached = True
trailing_stop_positive = 0.01
trailing_stop_positive_offset = 0.032
#Optimal timeframe for the strategy.
timeframe = '1m'
# Define the parameter spaces
buy_ema_short = IntParameter(3, 50, default=5)
buy_ema_long = IntParameter(15, 200, default=50)
# Run "populate_indicators()" only for new candle.
process_only_new_candles = False
# These values can be overridden in the "ask_strategy" section in the config.
use_sell_signal = True
sell_profit_only = False
ignore_roi_if_buy_signal = False
# Number of candles the strategy requires before producing valid signals
startup_candle_count: int = 30
# Optional order type mapping.
order_types = {
'buy': 'limit',
'sell': 'limit',
'stoploss': 'market',
'stoploss_on_exchange': False
}
# Optional order time in force.
order_time_in_force = {
'buy': 'gtc',
'sell': 'gtc'
}
plot_config = {
'main_plot': {
'tema': {},
'sar': {'color': 'white'},
},
'subplots': {
"MACD": {
'macd': {'color': 'blue'},
'macdsignal': {'color': 'orange'},
},
"RSI": {
'rsi': {'color': 'red'},
}
}
}
def informative_pairs(self):
"""
Define additional, informative pair/interval combinations to be cached from the exchange.
These pair/interval combinations are non-tradeable, unless they are part
of the whitelist as well.
For more information, please consult the documentation
:return: List of tuples in the format (pair, interval)
Sample: return [("ETH/USDT", "5m"),
("BTC/USDT", "15m"),
]
"""
return []
#hyperoptable parameters
#BUY PARAMETERS
buy_rsi = IntParameter(10, 60, default=47, space='buy') #RSI
buy_rsi_enabled = BooleanParameter(default=False, space='buy') #RSI
buy_adx = IntParameter(10, 46, default=22, space="buy") #ADX
buy_fastd = IntParameter(15, 45, default=25, space='buy') # Stochastic Fast
buy_fastk = IntParameter(15, 45, default=39, space='buy') # Stochastic Fast
buy_trigger = CategoricalParameter(['bb_upper_buy', 'bb_lower_buy', 'bb_middle_buy','wbb_upper_buy', 'wbb_lower_buy', 'wbb_middle_buy'], default = 'wbb_upper_buy', space = 'buy')
buy_adx_enabled = BooleanParameter(default=True, space='buy') #ADX
buy_fastk_enabled = BooleanParameter(default=False, space='buy') # Stochastic Fast
buy_fastd_enabled = BooleanParameter(default=True, space='buy') # Stochastic Fast
buy_rsi_enabled = BooleanParameter(default=False, space='buy') #RSI
#SELL PARAMETERS
sell_rsi = IntParameter(60, 80, default=71, space='sell') #RSI
sell_adx = IntParameter(50, 100, default=75, space='sell')#ADX
sell_fastd = IntParameter(50, 100, default=95, space='sell')# Stochastic Fast
sell_fastk = IntParameter(50, 100, default=70, space='sell')# Stochastic Fast
sell_adx_enabled = BooleanParameter(default=False, space='sell') #ADX
sell_fastd_enabled = BooleanParameter(default=False, space='sell') # Stochastic Fast
sell_fastk_enabled = BooleanParameter(default=False, space='sell') # Stochastic Fast
sell_rsi_enabled = BooleanParameter(default=True, space='sell') #RSI
sell_trigger = CategoricalParameter(['bb_lower_sell', 'bb_middle_sell','wbb_lower_sell', 'wbb_middle_sell'], default = 'wbb_middle_sell', space = 'sell')
#PROTECTIONS
stoploss = -0.237
timeframe = '1m'
# Define the parameter spaces
cooldown_lookback = IntParameter(2, 48, default=2, space="protection", optimize=True)
stop_duration = IntParameter(12, 200, default=102, space="protection", optimize=True)
use_stop_protection = BooleanParameter(default=True, space="protection", optimize=True)
@property
def protections(self):
prot = []
prot.append({
"method": "CooldownPeriod",
"stop_duration_candles": self.cooldown_lookback.value
})
if self.use_stop_protection.value:
prot.append({
"method": "StoplossGuard",
"lookback_period_candles": 24 * 3,
"trade_limit": 4,
"stop_duration_candles": self.stop_duration.value,
"only_per_pair": False
})
return prot
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Adds several different TA indicators to the given DataFrame
Performance Note: For the best performance be frugal on the number of indicators
you are using. Let uncomment only the indicator you are using in your strategies
or your hyperopt configuration, otherwise you will waste your memory and CPU usage.
:param dataframe: Dataframe with data from the exchange
:param metadata: Additional information, like the currently traded pair
:return: a Dataframe with all mandatory indicators for the strategies
"""
# Momentum Indicators
# ------------------------------------
# RSI
dataframe['rsi'] = ta.RSI(dataframe)
# # Stochastic Slow
stoch = ta.STOCH(dataframe)
dataframe['slowd'] = stoch['slowd']
dataframe['slowk'] = stoch['slowk']
# Stochastic Fast
stoch_fast = ta.STOCHF(dataframe)
dataframe['fastd'] = stoch_fast['fastd']
dataframe['fastk'] = stoch_fast['fastk']
# # Stochastic RSI
# Please read https://github.com/freqtrade/freqtrade/issues/2961 before using this.
# STOCHRSI is NOT aligned with tradingview, which may result in non-expected results.
stoch_rsi = ta.STOCHRSI(dataframe)
dataframe['fastd_rsi'] = stoch_rsi['fastd']
dataframe['fastk_rsi'] = stoch_rsi['fastk']
# Parabolic SAR
dataframe['sar'] = ta.SAR(dataframe)
# ADX
dataframe['adx'] = ta.ADX(dataframe)
# # Plus Directional Indicator / Movement
dataframe['plus_dm'] = ta.PLUS_DM(dataframe)
dataframe['plus_di'] = ta.PLUS_DI(dataframe)
# # Minus Directional Indicator / Movement
dataframe['minus_dm'] = ta.MINUS_DM(dataframe)
dataframe['minus_di'] = ta.MINUS_DI(dataframe)
# MACD
macd = ta.MACD(dataframe)
dataframe['macd'] = macd['macd']
dataframe['macdsignal'] = macd['macdsignal']
dataframe['macdhist'] = macd['macdhist']
# Bollinger Bands
bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=1)
dataframe['bb_lowerband'] = bollinger['lower']
dataframe['bb_middleband'] = bollinger['mid']
dataframe['bb_upperband'] = bollinger['upper']
dataframe["bb_percent"] = (
(dataframe["close"] - dataframe["bb_lowerband"]) /
(dataframe["bb_upperband"] - dataframe["bb_lowerband"])
)
# Bollinger Bands - Weighted (EMA based instead of SMA)
weighted_bollinger = qtpylib.weighted_bollinger_bands(
qtpylib.typical_price(dataframe), window=20, stds=2
)
dataframe["wbb_upperband"] = weighted_bollinger["upper"]
dataframe["wbb_lowerband"] = weighted_bollinger["lower"]
dataframe["wbb_middleband"] = weighted_bollinger["mid"]
dataframe["wbb_percent"] = (
(dataframe["close"] - dataframe["wbb_lowerband"]) /
(dataframe["wbb_upperband"] - dataframe["wbb_lowerband"])
)
dataframe["wbb_width"] = (
(dataframe["wbb_upperband"] - dataframe["wbb_lowerband"]) /
dataframe["wbb_middleband"]
)
# Awesome Oscillator
dataframe['ao'] = qtpylib.awesome_oscillator(dataframe)
# EMA - Exponential Moving Average
dataframe['ema3'] = ta.EMA(dataframe, timeperiod=3)
dataframe['ema5'] = ta.EMA(dataframe, timeperiod=5)
dataframe['ema10'] = ta.EMA(dataframe, timeperiod=10)
dataframe['ema21'] = ta.EMA(dataframe, timeperiod=21)
dataframe['ema50'] = ta.EMA(dataframe, timeperiod=50)
dataframe['ema100'] = ta.EMA(dataframe, timeperiod=100)
dataframe['ema200'] = ta.EMA(dataframe, timeperiod=200)
# # SMA - Simple Moving Average
dataframe['sma3'] = ta.SMA(dataframe, timeperiod=3)
dataframe['sma5'] = ta.SMA(dataframe, timeperiod=5)
dataframe['sma10'] = ta.SMA(dataframe, timeperiod=10)
dataframe['sma21'] = ta.SMA(dataframe, timeperiod=21)
dataframe['sma50'] = ta.SMA(dataframe, timeperiod=50)
dataframe['sma100'] = ta.SMA(dataframe, timeperiod=100)
# TEMA - Triple Exponential Moving Average
dataframe['tema'] = ta.TEMA(dataframe, timeperiod=9)
dataframe['tema50'] = ta.TEMA(dataframe, timeperiod=50)
dataframe['tema100'] = ta.TEMA(dataframe, timeperiod=100)
dataframe['tema200'] = ta.TEMA(dataframe, timeperiod=200)
# MFI
dataframe['mfi'] = ta.MFI(dataframe)
# Parabolic SAR
dataframe['sar'] = ta.SAR(dataframe)
# Calculate all ema_short values
frames = [dataframe]
for val in self.buy_ema_short.range:
frames.append(DataFrame({
f'ema_short_{val}': ta.EMA(dataframe, timeperiod=val)
}))
# Append columns to existing dataframe
merged_frame = pd.concat(frames, axis=1)
# Calculate all ema_long values
frames = [dataframe]
for val in self.buy_ema_long.range:
frames.append(DataFrame({
f'ema_long_{val}': ta.EMA(dataframe, timeperiod=val)
}))
# Append columns to existing dataframe
merged_frame = pd.concat(frames, axis=1)
# Cycle Indicator
# ------------------------------------
# # Chart type
# # ------------------------------------
# # Heikin Ashi Strategy
# heikinashi = qtpylib.heikinashi(dataframe)
# dataframe['ha_open'] = heikinashi['open']
# dataframe['ha_close'] = heikinashi['close']
# dataframe['ha_high'] = heikinashi['high']
# dataframe['ha_low'] = heikinashi['low']
return dataframe
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
conditions = []
#GUARDS & TRENDS
if self.buy_rsi_enabled.value:
conditions.append(dataframe['rsi'] > self.buy_rsi.value)
if self.buy_fastd_enabled.value:
conditions.append(dataframe['fastd'] < self.buy_fastd.value)
if self.buy_fastk_enabled.value:
conditions.append(dataframe['fastk'] < self.buy_fastk.value)
if self.buy_adx_enabled.value:
conditions.append(dataframe['adx'] > self.buy_adx.value)
#TRIGGERS
if self.buy_trigger.value == 'bb_lower_buy':
conditions.append(dataframe['close'] < dataframe['bb_lowerband'])
if self.buy_trigger.value =='bb_middle_buy':
conditions.append(dataframe ['close'] < dataframe['bb_middleband'])
if self.buy_trigger.value =='bb_upper_buy':
conditions.append(dataframe ['close'] < dataframe['bb_upperband'])
if self.buy_trigger.value == 'wbb_lower_buy':
conditions.append(dataframe['close'] < dataframe['wbb_lowerband'])
if self.buy_trigger.value =='wbb_middle_buy':
conditions.append(dataframe ['close'] < dataframe['wbb_middleband'])
if self.buy_trigger.value =='wbb_upper_buy':
conditions.append(dataframe ['close'] < dataframe['wbb_upperband'])
conditions.append(qtpylib.crossed_above(
dataframe[f'ema_short_{self.buy_ema_short.value}'], dataframe[f'ema_long_{self.buy_ema_long.value}']
))
conditions.append (dataframe['volume'] > 0)
if conditions:
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'buy'] = 1
return dataframe
def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
conditions = []
#GUARDS & TRENDS
if self.sell_rsi_enabled.value:
conditions.append(dataframe['rsi'] < self.sell_rsi.value)
if self.sell_fastd_enabled.value:
conditions.append(dataframe['fastd'] > self.sell_fastd.value)
if self.sell_fastk_enabled.value:
conditions.append(dataframe['fastk'] > self.sell_fastk.value)
if self.sell_adx_enabled.value:
conditions.append(dataframe['adx'] < self.sell_adx.value)
#TRIGGERS
if self.sell_trigger.value == 'bb_upper_sell':
conditions.append(dataframe['close'] > dataframe['bb_upperband'])
if self.sell_trigger.value =='bb_middle_sell':
conditions.append(dataframe ['close'] > dataframe['bb_middleband'])
if self.sell_trigger.value =='bb_low_sell':
conditions.append(dataframe ['close'] > dataframe['bb_middleband'])
if self.sell_trigger.value == 'wbb_upper_sell':
conditions.append(dataframe['close'] > dataframe['wbb_upperband'])
if self.sell_trigger.value =='wbb_middle_sell':
conditions.append(dataframe ['close'] > dataframe['wbb_middleband'])
if self.sell_trigger.value =='wbb_low_sell':
conditions.append(dataframe ['close'] > dataframe['wbb_middleband'])
conditions.append(qtpylib.crossed_above(
dataframe[f'ema_long_{self.buy_ema_long.value}'], dataframe[f'ema_short_{self.buy_ema_short.value}']
))
conditions.append (dataframe['volume'] > 0)
if conditions:
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'sell'] = 1
return dataframe`
I'd really appreciate if you look it through and tell me what's wrong with it... Thank you in advance for all your advices.
I have this mistake, when I run hyperopt:
freqtrade - ERROR - Fatal exception! joblib.externals.loky.process_executor._RemoteTraceback: """ Traceback (most recent call last): File "/root/freqtrade/.env/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3361, in get_loc return self._engine.get_loc(casted_key) File "pandas/_libs/index.pyx", line 76, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'ema_short_21'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/freqtrade/.env/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 436, in _process_worker
r = call_item()
File "/root/freqtrade/.env/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 288, in call
return self.fn(*self.args, self.kwargs)
File "/root/freqtrade/.env/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 595, in call
return self.func(*args, *kwargs)
File "/root/freqtrade/.env/lib/python3.8/site-packages/joblib/parallel.py", line 262, in call
return [func(args, kwargs)
File "/root/freqtrade/.env/lib/python3.8/site-packages/joblib/parallel.py", line 262, in
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/root/freqtrade/freqtrade/main.py", line 37, in main return_code = args'func' File "/root/freqtrade/freqtrade/commands/optimize_commands.py", line 103, in start_hyperopt hyperopt.start() File "/root/freqtrade/freqtrade/optimize/hyperopt.py", line 476, in start f_val = self.run_optimizer_parallel(parallel, asked, i) File "/root/freqtrade/freqtrade/optimize/hyperopt.py", line 390, in run_optimizer_parallel return parallel(delayed( File "/root/freqtrade/.env/lib/python3.8/site-packages/joblib/parallel.py", line 1056, in call self.retrieve() File "/root/freqtrade/.env/lib/python3.8/site-packages/joblib/parallel.py", line 935, in retrieve self._output.extend(job.get(timeout=self.timeout)) File "/root/freqtrade/.env/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 542, in wrap_future_result return future.result(timeout=timeout) File "/usr/lib/python3.8/concurrent/futures/_base.py", line 444, in result return self.get_result() File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in get_result raise self._exception KeyError: 'ema_short_21' (.env) root@ubuntu-4gb-hel1-1:~/freqtrade#
And I also have this mistake, when I try to run backtest:
freqtrade - ERROR - Fatal exception! Traceback (most recent call last): File "/root/freqtrade/.env/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3361, in get_loc return self._engine.get_loc(casted_key) File "pandas/_libs/index.pyx", line 76, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'ema_short_5'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/root/freqtrade/freqtrade/main.py", line 37, in main return_code = args'func' File "/root/freqtrade/freqtrade/commands/optimize_commands.py", line 54, in start_backtesting backtesting.start() File "/root/freqtrade/freqtrade/optimize/backtesting.py", line 672, in start min_date, max_date = self.backtest_one_strategy(strat, data, timerange) File "/root/freqtrade/freqtrade/optimize/backtesting.py", line 643, in backtest_one_strategy results = self.backtest( File "/root/freqtrade/freqtrade/optimize/backtesting.py", line 519, in backtest data: Dict = self._get_ohlcv_as_lists(processed) File "/root/freqtrade/freqtrade/optimize/backtesting.py", line 267, in _get_ohlcv_as_lists self.strategy.advise_buy(pair_data, {'pair': pair}), {'pair': pair}).copy() File "/root/freqtrade/freqtrade/strategy/interface.py", line 874, in advise_buy return self.populate_buy_trend(dataframe, metadata) File "/root/freqtrade/user_data/strategies/BBRSIADXconfig2v31.py", line 376, in populate_buy_trend dataframe[f'emashort{self.buy_ema_short.value}'], dataframe[f'emalong{self.buy_ema_long.value}'] File "/root/freqtrade/.env/lib/python3.8/site-packages/pandas/core/frame.py", line 3458, in getitem indexer = self.columns.get_loc(key) File "/root/freqtrade/.env/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc raise KeyError(key) from err KeyError: 'ema_short_5
Hello @straj-granicy have you find a solution to this problem because i have the same and would like to learn how to resolve? Thanks
Well the error above is simple - the assignment of pd.concat()
creates a new variable (twice) - but doesn't reuse that variable later - so everything done in the loops is "forgotten" once the method exits.
Sorry to be a newbee in python so the solution is simply : dataframes = pd.concat(frames, axis=1) ??
singular (dataframe, not dataframes) - but yes.
dataframe = pd.concat(frames, axis=1)
Alternatively, you can also use merged_frame
going forward - the important part is that what you generate will also be used going forward - the name is just that - a name.
Thanks a lot xmatthias !!! It's a problem i was trying to solve since two month...
Old newbees are really slow...
Trying to optimize the Tema parameter, as described in tutorial. But getting the following error (qtpylib.crossed_above(dataframe[f'temamiddle{self.buy_tema_middle.value}'],dataframe[f'temalong{self.buy_tema_long.value}']))& File "/root/freqtrade/.env/lib/python3.8/site-packages/pandas/core/frame.py", line 3458, in getitem indexer = self.columns.get_loc(key) File "/root/freqtrade/.env/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc raise KeyError(key) from err KeyError: 'tema_middle_100'
Could someone advise me what could be the problem? Thanks.