search

iterativv / NostalgiaForInfinity

Trading strategy for the Freqtrade crypto bot
GNU General Public License v3.0
2.04k stars 506 forks source link

ERROR when Backtesting #253

Closed Matsmann closed 1 year ago

Matsmann commented 2 years ago

2022-10-09 19:24:03,381 - freqtrade.optimize.backtesting - INFO - Loading data from 2021-07-30 08:00:00 up to 2022-10-09 19:00:00 (436 days). 2022-10-09 19:24:03,381 - freqtrade.optimize.backtesting - INFO - Dataload complete. Calculating indicators 2022-10-09 19:24:03,385 - freqtrade.optimize.backtesting - WARNING - Backtest result caching disabled due to use of open-ended timerange. 2022-10-09 19:24:03,385 - freqtrade.optimize.backtesting - INFO - Running backtesting for Strategy NFIX 2022-10-09 19:24:03,385 - freqtrade.strategy.hyper - INFO - No params for protection found, using default values. 2022-10-09 19:24:14,920 - freqtrade.data.history.idatahandler - WARNING - AAVE/USDT, spot, 1d, data starts at 2020-10-15 00:00:00 2022-10-09 19:24:35,197 - freqtrade.data.history.idatahandler - WARNING - AVA/USDT, spot, 1d, data starts at 2020-11-24 00:00:00 2022-10-09 19:24:40,128 - freqtrade.data.history.idatahandler - WARNING - AVAX/USDT, spot, 1d, data starts at 2020-09-22 00:00:00 2022-10-09 19:24:45,009 - freqtrade.data.history.idatahandler - WARNING - AXS/USDT, spot, 1d, data starts at 2020-11-04 00:00:00 2022-10-09 19:24:49,894 - freqtrade.data.history.idatahandler - WARNING - CAKE/USDT, spot, 1d, data starts at 2021-02-19 00:00:00 2022-10-09 19:24:54,786 - freqtrade.data.history.idatahandler - WARNING - CELO/USDT, spot, 1d, data starts at 2021-01-05 00:00:00 2022-10-09 19:24:59,877 - freqtrade.data.history.idatahandler - WARNING - CHR/USDT, spot, 1d, data starts at 2020-05-07 00:00:00 2022-10-09 19:25:04,948 - freqtrade.data.history.idatahandler - WARNING - CKB/USDT, spot, 1d, data starts at 2021-01-26 00:00:00 2022-10-09 19:25:09,943 - freqtrade.data.history.idatahandler - WARNING - COMP/USDT, spot, 1d, data starts at 2020-06-25 00:00:00 2022-10-09 19:25:20,138 - freqtrade.data.history.idatahandler - WARNING - CRV/USDT, spot, 1d, data starts at 2020-08-15 00:00:00 2022-10-09 19:25:25,185 - freqtrade.data.history.idatahandler - WARNING - DGB/USDT, spot, 1d, data starts at 2020-07-20 00:00:00 2022-10-09 19:25:30,281 - freqtrade.data.history.idatahandler - WARNING - DOT/USDT, spot, 1d, data starts at 2020-08-18 00:00:00 2022-10-09 19:25:35,552 - freqtrade.data.history.idatahandler - WARNING - DYDX/USDT, spot, 1d, data starts at 2021-09-09 00:00:00 2022-10-09 19:25:35,637 - freqtrade.data.history.idatahandler - WARNING - DYDX/USDT, spot, 1h, data starts at 2021-09-09 02:00:00 2022-10-09 19:25:35,984 - freqtrade.data.history.idatahandler - WARNING - DYDX/USDT, spot, 15m, data starts at 2021-09-09 02:00:00 2022-10-09 19:25:50,494 - freqtrade.data.history.idatahandler - WARNING - FIL/USDT, spot, 1d, data starts at 2020-10-15 00:00:00 2022-10-09 19:26:00,685 - freqtrade.data.history.idatahandler - WARNING - GRT/USDT, spot, 1d, data starts at 2020-12-17 00:00:00 2022-10-09 19:26:05,671 - freqtrade.data.history.idatahandler - WARNING - ICP/USDT, spot, 1d, data starts at 2021-05-11 00:00:00 2022-10-09 19:26:32,281 - freqtrade.data.history.idatahandler - WARNING - MANA/USDT, spot, 1d, data starts at 2020-08-06 00:00:00 2022-10-09 19:26:37,415 - freqtrade.data.history.idatahandler - WARNING - MASK/USDT, spot, 1d, data starts at 2021-05-25 00:00:00 2022-10-09 19:26:47,644 - freqtrade.data.history.idatahandler - WARNING - MOVR/USDT, spot, 1d, data starts at 2021-11-08 00:00:00 2022-10-09 19:26:47,747 - freqtrade.data.history.idatahandler - WARNING - MOVR/USDT, spot, 1h, data starts at 2021-11-08 06:00:00 2022-10-09 19:26:47,991 - freqtrade.data.history.idatahandler - WARNING - MOVR/USDT, spot, 15m, data starts at 2021-11-08 06:00:00 2022-10-09 19:26:51,405 - freqtrade.data.history.idatahandler - WARNING - NEAR/USDT, spot, 1d, data starts at 2020-10-14 00:00:00 2022-10-09 19:27:01,833 - freqtrade.data.history.idatahandler - WARNING - ORN/USDT, spot, 1d, data starts at 2020-09-30 00:00:00 2022-10-09 19:27:06,908 - freqtrade.data.history.idatahandler - WARNING - PHA/USDT, spot, 1d, data starts at 2021-06-25 00:00:00 2022-10-09 19:27:12,133 - freqtrade.data.history.idatahandler - WARNING - PYR/USDT, spot, 1d, data starts at 2021-11-26 00:00:00 2022-10-09 19:27:12,206 - freqtrade.data.history.idatahandler - WARNING - PYR/USDT, spot, 1h, data starts at 2021-11-26 08:00:00 2022-10-09 19:27:12,433 - freqtrade.data.history.idatahandler - WARNING - PYR/USDT, spot, 15m, data starts at 2021-11-26 08:00:00 2022-10-09 19:27:15,773 - freqtrade.data.history.idatahandler - WARNING - QI/USDT, spot, 1d, data starts at 2021-11-15 00:00:00 2022-10-09 19:27:15,842 - freqtrade.data.history.idatahandler - WARNING - QI/USDT, spot, 1h, data starts at 2021-11-15 08:00:00 2022-10-09 19:27:16,077 - freqtrade.data.history.idatahandler - WARNING - QI/USDT, spot, 15m, data starts at 2021-11-15 08:00:00 2022-10-09 19:27:19,725 - freqtrade.data.history.idatahandler - WARNING - QNT/USDT, spot, 1d, data starts at 2021-07-29 00:00:00 2022-10-09 19:27:19,812 - freqtrade.data.history.idatahandler - WARNING - QNT/USDT, spot, 1h, data starts at 2021-07-29 06:00:00 2022-10-09 19:27:20,203 - freqtrade.data.history.idatahandler - WARNING - QNT/USDT, spot, 15m, data starts at 2021-07-29 06:00:00 ERROR: 137

Hi

keeps getting ERROR messages and killing when doing backtesting. Any adwise on what to do ?

contix93 commented 2 years ago

Running out of memory, you should try to backtest a shorter period

Matsmann commented 2 years ago

Thanks that helped after shortning down to 3months. Has anyone experienced the same problems when livetrading ?

iterativv commented 2 years ago

Thanks that helped after shortning down to 3months. Has anyone experienced the same problems when livetrading ?

In live doesn't matter, RAM is never the issue. 500MB normally is enough.

SeanTolstoyevski commented 1 year ago

hi @iterativv , :)

I have a similar but not identical issue to this. I'm using Windows 11 and running it with Docker.

First I download the data:

docker-compose run --rm freqtrade download-data -c "user_data/config.json" --timerange 20230101-

non problem. continue.

then I want to run backtesting:

docker-compose run --rm freqtrade backtesting --stake-amount 100 --strategy NostalgiaForInfinityX2

Bingo! an error:

...>docker-compose run --rm freqtrade backtesting --stake-amount 100 --strategy NostalgiaForInfinityX2
2023-02-12 23:25:50,406 - freqtrade - INFO - freqtrade 2022.9.1
2023-02-12 23:25:50,417 - freqtrade.configuration.load_config - INFO - Using config: user_data/config.json ...
2023-02-12 23:25:50,422 - freqtrade.loggers - INFO - Verbosity set to 0
2023-02-12 23:25:50,422 - freqtrade.configuration.configuration - WARNING - `force_entry_enable` RPC message enabled.
2023-02-12 23:25:50,422 - freqtrade.configuration.configuration - INFO - Using max_open_trades: 6 ...
2023-02-12 23:25:50,422 - freqtrade.configuration.configuration - INFO - Parameter --stake-amount detected, overriding stake_amount to: 100.0 ...
2023-02-12 23:25:50,609 - freqtrade.configuration.configuration - INFO - Using user-data directory: /freqtrade/user_data ...
2023-02-12 23:25:50,614 - freqtrade.configuration.configuration - INFO - Using data directory: /freqtrade/user_data/data/binance ...
2023-02-12 23:25:50,614 - freqtrade.configuration.configuration - INFO - Parameter --cache=day detected ...
2023-02-12 23:25:50,614 - freqtrade.configuration.check_exchange - INFO - Checking exchange...
2023-02-12 23:25:50,626 - freqtrade.configuration.check_exchange - INFO - Exchange "binance" is officially supported by the Freqtrade development team.
2023-02-12 23:25:50,627 - freqtrade.configuration.configuration - INFO - Using pairlist from configuration.
2023-02-12 23:25:50,627 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2023-02-12 23:25:50,631 - freqtrade.commands.optimize_commands - INFO - Starting freqtrade in Backtesting mode
2023-02-12 23:25:50,632 - freqtrade.exchange.exchange - INFO - Instance is running with dry_run enabled
2023-02-12 23:25:50,632 - freqtrade.exchange.exchange - INFO - Using CCXT 1.93.98
2023-02-12 23:25:50,648 - freqtrade.exchange.exchange - INFO - Using Exchange "Binance"
2023-02-12 23:25:52,919 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Binance'...
2023-02-12 23:25:53,062 - freqtrade.resolvers.iresolver - INFO - Using resolved strategy NostalgiaForInfinityX2 from '/freqtrade/user_data/strategies/NostalgiaForInfinityX2.py'...
2023-02-12 23:25:53,065 - freqtrade.strategy.hyper - INFO - Found no parameter file.
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'order_types' with value in config file: {'entry': 'limit', 'exit': 'limit', 'emergency_exit': 'limit', 'force_entry': 'limit', 'force_exit': 'limit', 'stoploss': 'limit', 'stoploss_on_exchange': False, 'stoploss_on_exchange_interval': 60}.
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'stake_currency' with value in config file: USDT.
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'stake_amount' with value in config file: 100.0.
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'unfilledtimeout' with value in config file: {'entry': 2, 'exit': 2, 'exit_timeout_count': 0, 'unit': 'minutes'}.
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using minimal_roi: {'0': 100.0}
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using timeframe: 5m
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stoploss: -0.99
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop: False
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop_positive: 0.01
2023-02-12 23:25:53,066 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop_positive_offset: 0.03
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_only_offset_is_reached: True
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using use_custom_stoploss: False
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using process_only_new_candles: True
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using order_types: {'entry': 'limit', 'exit': 'limit', 'emergency_exit': 'limit', 'force_entry': 'limit', 'force_exit': 'limit', 'stoploss': 'limit', 'stoploss_on_exchange': False, 'stoploss_on_exchange_interval': 60}
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using order_time_in_force: {'entry': 'GTC', 'exit': 'GTC'}
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stake_currency: USDT
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stake_amount: 100.0
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using protections: []
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using startup_candle_count: 480
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using unfilledtimeout: {'entry': 2, 'exit': 2, 'exit_timeout_count': 0, 'unit': 'minutes'}
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using use_exit_signal: True
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using exit_profit_only: False
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using ignore_roi_if_entry_signal: True
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using exit_profit_offset: 0.0
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using disable_dataframe_checks: False
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using ignore_buying_expired_candle_after: 0
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using position_adjustment_enable: True
2023-02-12 23:25:53,067 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using max_entry_position_adjustment: -1
2023-02-12 23:25:53,067 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2023-02-12 23:25:53,071 - freqtrade.resolvers.iresolver - INFO - Using resolved pairlist StaticPairList from '/freqtrade/freqtrade/plugins/pairlist/StaticPairList.py'...
2023-02-12 23:25:53,100 - freqtrade.data.history.history_utils - INFO - Using indicator startup period: 480 ...
2023-02-12 23:25:53,376 - freqtrade.optimize.backtesting - INFO - Loading data from 2023-01-01 00:00:00 up to 2023-02-12 23:10:00 (42 days).
2023-02-12 23:25:53,376 - freqtrade.configuration.timerange - WARNING - Moving start-date by 480 candles to account for startup time.
2023-02-12 23:25:53,376 - freqtrade.optimize.backtesting - INFO - Dataload complete. Calculating indicators
2023-02-12 23:25:53,387 - freqtrade.optimize.backtesting - WARNING - Backtest result caching disabled due to use of open-ended timerange.
2023-02-12 23:25:53,387 - freqtrade.optimize.backtesting - INFO - Running backtesting for Strategy NostalgiaForInfinityX2
2023-02-12 23:25:53,388 - freqtrade.strategy.hyper - INFO - No params for sell found, using default values.
2023-02-12 23:25:53,388 - freqtrade.strategy.hyper - INFO - No params for protection found, using default values.
2023-02-12 23:25:53,441 - freqtrade.data.history.idatahandler - WARNING - No history for BTC/USDT, spot, 15m found. Use `freqtrade download-data` to download the data
2023-02-12 23:25:53,442 - freqtrade.data.dataprovider - WARNING - No data found for (BTC/USDT, 15m, ).
2023-02-12 23:25:53,443 - freqtrade - ERROR - Fatal exception!
Traceback (most recent call last):
  File "/freqtrade/freqtrade/main.py", line 39, in main
    return_code = args['func'](args)
  File "/freqtrade/freqtrade/commands/optimize_commands.py", line 58, in start_backtesting
    backtesting.start()
  File "/freqtrade/freqtrade/optimize/backtesting.py", line 1332, in start
    min_date, max_date = self.backtest_one_strategy(strat, data, timerange)
  File "/freqtrade/freqtrade/optimize/backtesting.py", line 1230, in backtest_one_strategy
    preprocessed = self.strategy.advise_all_indicators(data)
  File "/freqtrade/freqtrade/strategy/interface.py", line 1205, in advise_all_indicators
    return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy()
  File "/freqtrade/freqtrade/strategy/interface.py", line 1205, in <dictcomp>
    return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy()
  File "/freqtrade/freqtrade/strategy/interface.py", line 1223, in advise_indicators
    return self.populate_indicators(dataframe, metadata)
  File "/freqtrade/user_data/strategies/NostalgiaForInfinityX2.py", line 4704, in populate_indicators
    btc_informative = self.btc_info_switcher(btc_info_pair, btc_info_timeframe, metadata)
  File "/freqtrade/user_data/strategies/NostalgiaForInfinityX2.py", line 4686, in btc_info_switcher
    return self.btc_info_15m_indicators(btc_info_pair, btc_info_timeframe, metadata)
  File "/freqtrade/user_data/strategies/NostalgiaForInfinityX2.py", line 4636, in btc_info_15m_indicators
    btc_info_15m['rsi_14'] = ta.RSI(btc_info_15m, timeperiod=14)
  File "talib/_abstract.pxi", line 445, in talib._ta_lib.Function.__call__
  File "talib/_abstract.pxi", line 492, in talib._ta_lib.Function.__call_function
  File "/home/ftuser/.local/lib/python3.10/site-packages/talib/__init__.py", line 64, in wrapper
    result = func(*_args, **_kwds)
  File "talib/_func.pxi", line 4377, in talib._ta_lib.RSI
  File "talib/_func.pxi", line 68, in talib._ta_lib.check_begidx1
Exception: inputs are all NaN

my config.json file:

{
    "max_open_trades": 6,
    "stake_currency": "USDT",
    "stake_amount": "unlimited",
    "tradable_balance_ratio": 0.90,
    "fiat_display_currency": "USD",
    "dry_run": true,
    "dry_run_wallet": 200,
    "force_entry_enable": true,
    //    "trading_mode": "futures",
    "unfilledtimeout": {
        "entry": 2,
        "exit": 2,
        "exit_timeout_count": 0,
        "unit": "minutes"
    },
    "order_types": {
        "entry": "limit",
        "exit": "limit",
        "emergency_exit": "limit",
        "force_entry": "limit",
        "force_exit": "limit",
        "stoploss": "limit",
        "stoploss_on_exchange": false,
        "stoploss_on_exchange_interval": 60
    },
    "entry_pricing": {
        "price_side": "other",
        "use_order_book": false,
        "order_book_top": 1,
        "price_last_balance": 0.0,
        "check_depth_of_market": {
            "enabled": false,
            "bids_to_ask_delta": 1
        },
    },
    "exit_pricing": {
        "price_side": "other",
        "use_order_book": false,
        "order_book_top": 1,
        "price_last_balance": 0.0,
    },
    "exchange": {
        "name": "binance",
        "key": "",
        "secret": "",
        "ccxt_config": {},
        "ccxt_async_config": {},
        "pair_whitelist": [
            "BTC/USDT",
            "XRP/USDT",
            "MATIC/USDT",
            "NEAR/USDT",
            "DOT/USDT"
        ],
        "pair_blacklist": [
            "(BNB)/.*",
            ".*(_PREMIUM|BEAR|BULL|HALF|HEDGE|UP|DOWN|[1235][SL])/.*",
            "(AUD|BRZ|CAD|CHF|EUR|GBP|HKD|IDRT|JPY|NGN|RUB|SGD|TRY|UAH|USD|ZAR)/.*",
            "(BUSD|CUSD|CUSDT|DAI|PAXG|SUSD|TUSD|USDC|USDN|USDP|USDT|VAI|UST|USTC|AUSD)/.*",
            "(ACM|AFA|ALA|ALL|ALPINE|APL|ASR|ATM|BAR|CAI|CHZ|CITY|FOR|GAL|GOZ|IBFK|JUV|LEG|LOCK-1|NAVI|NMR|NOV|PFL|PSG|ROUSH|STV|TH|TRA|UCH|UFC|YBO)/.*",
            "(1EARTH|ILA|BOBA|CTXC|CWAR|HBAR|NMR|OMG|ONG|ARDR|DMTR|MLS|TORN|ANC|LUNA|BTS|QKC|COS|ACA|FTT|SRM|YFII|SNM|BNX|WAVES|MIR)/.*"
        ]
    },
    "pairlists": [
        {
            "method": "StaticPairList",
        }
    ],
    "edge": {
        "enabled": false,
        "process_throttle_secs": 3600,
        "calculate_since_number_of_days": 7,
        "allowed_risk": 0.06,
        "stoploss_range_min": -0.01,
        "stoploss_range_max": -2.5,
        "stoploss_range_step": -0.01,
        "minimum_winrate": 0.60,
        "minimum_expectancy": 0.20,
        "min_trade_number": 10,
        "max_trade_duration_minute": 1,
        "remove_pumps": false
    },
    "api_server": {
        "enabled": true,
        "listen_ip_address": "0.0.0.0",
        "listen_port": 8080,
        "verbosity": "error",
        "enable_openapi": false,
        "jwt_secret_key": "6f6bd12573421fb4a8221a5428edaf9cc01ad91409d4ad8e65ba20b944799a5c",
        "ws_token": "Da-_0zzjyi7sAdaoBEcnb07MDkH1FH3ThQ",
        "CORS_origins": [],
        "username": "freqtrader",
        "password": "123"
    },
    "bot_name": "freqtrade",
    "initial_state": "running",
    "force_entry_enable": true,
    "internals": {
        "process_throttle_secs": 5
    }
}
sebastianmacias commented 1 year ago

I had the same issue and fixed it by redownloading the data #276.

nottjake commented 1 year ago

Yes redownloading the data making sure it keeps the exact same candle duration and timeframe, you may have conflicting downloaded data with your timeframe for backtesting.

iterativv commented 1 year ago

This is too old already. If there are still issue, please, don't hesitate to open a new issue.