search

iterativv / NostalgiaForInfinity

Trading strategy for the Freqtrade crypto bot
GNU General Public License v3.0
1.96k stars 486 forks source link

Unable to do hyperopt and backtest #155

Closed huytrvan closed 2 years ago

huytrvan commented 2 years ago

Describe issue

Hi @iterativv ,

I'm not able to hyperopt any NFI strategy. I've tried both on my pc and on AWS instance (both using docker) but no luck.

Hyperopting any NFI strategy will always give this error: ERROR - The 'buy' space is included into the hyperoptimization but no parameter for this space was not found in your Strategy. Please make sure to have parameters for this space enabled for optimization or remove the 'buy' space from hyperoptimization.

Note:

If your need further info I would like to provide

Steps taken to troubleshoot:

As the error suggests, I suspect that some parameters are missing from 'buy' space, so I added all buy and sell hyperopt parameters to buy_params and sell_params correspondingly.

Like this:

buy_params = {
        #############
        # Enable/Disable conditions
        "buy_condition_1_enable": True,
        "buy_condition_2_enable": True,
        "buy_condition_3_enable": True,
        "buy_condition_4_enable": True,
        "buy_condition_5_enable": True,
        "buy_condition_6_enable": True,
        "buy_condition_7_enable": True,
        "buy_condition_8_enable": True,
        "buy_condition_9_enable": True,
        "buy_condition_10_enable": True,
        "buy_condition_11_enable": True,
        "buy_condition_12_enable": True,
        "buy_condition_13_enable": True,
        "buy_condition_14_enable": True,
        "buy_condition_15_enable": True,
        "buy_condition_16_enable": True,
        "buy_condition_17_enable": True,
        "buy_condition_18_enable": True,
        "buy_condition_19_enable": True,
        "buy_condition_20_enable": True,
        "buy_condition_21_enable": True,
        "buy_condition_22_enable": True,
        "buy_condition_23_enable": True,
        "buy_condition_24_enable": True,
        "buy_condition_25_enable": True,
        "buy_condition_26_enable": True,
        #############
        "buy_01_protection__ema_fast": False,
        "buy_01_protection__ema_fast_len": "26",
        "buy_01_protection__ema_slow": True,
        "buy_01_protection__ema_slow_len": "100",
        "buy_01_protection__close_above_ema_fast": False,
        "buy_01_protection__close_above_ema_fast_len": "200",
        "buy_01_protection__close_above_ema_slow": False,
        "buy_01_protection__close_above_ema_slow_len": "200",
        "buy_01_protection__sma200_rising": True,
        "buy_01_protection__sma200_rising_val": "28",
        "buy_01_protection__sma200_1h_rising": False,
        "buy_01_protection__sma200_1h_rising_val": "50",
        "buy_01_protection__safe_dips": True,
        "buy_01_protection__safe_dips_type": "80",
        "buy_01_protection__safe_pump": True,
        "buy_01_protection__safe_pump_type": "70",
        "buy_01_protection__safe_pump_period": "24",
        #############
        [... more ...]
}

System

Config.json

{
    "max_open_trades": 10,
    "stake_currency": "USDT",
    "stake_amount": "unlimited",
    "tradable_balance_ratio": 0.99,
    "fiat_display_currency": "USD",
    "timeframe": "15m",
    "dry_run": true,
    "dry_run_wallet": 750,
    "cancel_open_orders_on_exit": false,
    "unfilledtimeout": {
        "buy": 10,
        "sell": 30,
        "unit": "minutes"
    },
    "bid_strategy": {
        "price_side": "bid",
        "ask_last_balance": 0.0,
        "use_order_book": true,
        "order_book_top": 1,
        "check_depth_of_market": {
            "enabled": false,
            "bids_to_ask_delta": 1
        }
    },
    "ask_strategy": {
        "price_side": "ask",
        "use_order_book": true,
        "order_book_top": 1
    },
    "exchange": {
        "name": "binance",
        "key": "",
        "secret": "",
        "ccxt_config": {"enableRateLimit": true},
        "ccxt_async_config": {
            "enableRateLimit": true,
            "rateLimit": 200
        },
        "pair_whitelist": [
            "BTC/USDT",
            "ETH/USDT",
            "SOL/USDT",
            "ADA/USDT",
            "XRP/USDT",
            "DOT/USDT",
            "DOGE/USDT",
            "SHIB/USDT",
            "LUNA/USDT",
            "AVAX/USDT",
            "LTC/USDT",
            "LINK/USDT",
            "UNI/USDT",
            "BCH/USDT",
            "ALGO/USDT",
            "MATIC/USDT",
            "VET/USDT",
            "XLM/USDT",
            "AXS/USDT",
            "ICP/USDT",
            "TRX/USDT",
            "FIL/USDT",
            "ETC/USDT",
            "ATOM/USDT",
            "THETA/USDT",
            "FTT/USDT",
            "FTM/USDT",
            "HBAR/USDT",
            "EGLD/USDT",
            "NEAR/USDT",
            "MANA/USDT",
            "HNT/USDT",
            "XTZ/USDT",
            "XMR/USDT",
            "EOS/USDT",
            "CAKE/USDT",
            "AAVE/USDT",
            "FLOW/USDT",
            "KLAY/USDT",
            "XEC/USDT",
            "LRC/USDT",
            "KSM/USDT",
            "RUNE/USDT",
            "NEO/USDT",
            "CHZ/USDT",
            "QNT/USDT",
            "MKR/USDT",
            "ONE/USDT",
            "STX/USDT",
            "ZEC/USDT",
            "ENJ/USDT",
            "SAND/USDT",
            "WAVES/USDT",
            "BTT/USDT",
            "AR/USDT",
            "DASH/USDT",
            "HOT/USDT",
            "COMP/USDT",
            "IOTX/USDT",
            "CELO/USDT",
            "TFUEL/USDT",
            "CRV/USDT",
            "BAT/USDT",
            "XEM/USDT",
            "OMG/USDT",
            "ICX/USDT",
            "QTUM/USDT",
            "DCR/USDT",
            "MINA/USDT",
            "SUSHI/USDT",
            "LPT/USDT",
            "RVN/USDT",
            "ZIL/USDT",
            "AUDIO/USDT",
            "YFI/USDT",
            "PERP/USDT",
            "BTG/USDT",
            "SNX/USDT",
            "ZEN/USDT",
            "WAXP/USDT"
        ],
        "pair_blacklist": [
            "BNB/.*"
        ]
    },
    "pairlists": [
        {"method": "StaticPairList"},
        {"method": "AgeFilter",
        "min_days_listed": 30
        }
    ],
    "edge": {
        "enabled": true,
        "process_throttle_secs": 43200,
        "calculate_since_number_of_days": 183,
        "allowed_risk": 0.01,
        "stoploss_range_min": -0.01,
        "stoploss_range_max": -0.1,
        "stoploss_range_step": -0.01,
        "minimum_winrate": 0.20,
        "minimum_expectancy": 0.6,
        "min_trade_number": 10,
        "max_trade_duration_minute": 1440,
        "remove_pumps": false
    },

[... more ...]

Log

$ docker-compose run freqtrade hyperopt -s NostalgiaForInfinityNextGen -d user_data/data/binance/ --hyperopt-loss SharpeHyperOptLoss --spaces buy sell
Creating freqtrade_wealthy1_bot_freqtrade_run ...
Creating freqtrade_wealthy1_bot_freqtrade_run ... done
2021-11-15 08:05:31,123 - freqtrade.configuration.configuration - INFO - Using config: user_data/config.json ...
2021-11-15 08:05:31,168 - freqtrade.loggers - INFO - Verbosity set to 0
2021-11-15 08:05:31,169 - freqtrade.configuration.configuration - INFO - Using max_open_trades: 10 ...
2021-11-15 08:05:32,604 - freqtrade.configuration.configuration - INFO - Using user-data directory: /freqtrade/user_data ...
2021-11-15 08:05:32,652 - freqtrade.configuration.configuration - INFO - Using data directory: user_data/data/binance ...
2021-11-15 08:05:32,654 - freqtrade.configuration.configuration - INFO - Parameter --epochs detected ... Will run Hyperopt with for 100 epochs ...
2021-11-15 08:05:32,655 - freqtrade.configuration.configuration - INFO - Parameter -s/--spaces detected: ['buy', 'sell']
2021-11-15 08:05:32,656 - freqtrade.configuration.configuration - INFO - Parameter -j/--job-workers detected: -1
2021-11-15 08:05:32,658 - freqtrade.configuration.configuration - INFO - Parameter --min-trades detected: 1
2021-11-15 08:05:32,659 - freqtrade.configuration.configuration - INFO - Using Hyperopt loss class name: SharpeHyperOptLoss
2021-11-15 08:05:32,670 - freqtrade.configuration.check_exchange - INFO - Checking exchange...
2021-11-15 08:05:32,927 - freqtrade.configuration.check_exchange - INFO - Exchange "binance" is officially supported by the Freqtrade development team.
2021-11-15 08:05:32,929 - freqtrade.configuration.configuration - INFO - Using pairlist from configuration.
2021-11-15 08:05:32,931 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2021-11-15 08:05:32,979 - freqtrade.commands.optimize_commands - INFO - Starting freqtrade in Hyperopt mode
2021-11-15 08:05:33,004 - filelock - INFO - Lock 139682979045968 acquired on /freqtrade/user_data/hyperopt.lock
2021-11-15 08:05:33,006 - freqtrade.exchange.exchange - INFO - Instance is running with dry_run enabled
2021-11-15 08:05:33,006 - freqtrade.exchange.exchange - INFO - Using CCXT 1.57.3
2021-11-15 08:05:33,007 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'enableRateLimit': True}
2021-11-15 08:05:33,187 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'enableRateLimit': True, 'rateLimit': 200}
2021-11-15 08:05:33,353 - freqtrade.exchange.exchange - INFO - Using Exchange "Binance"
2021-11-15 08:05:43,125 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Binance'...
2021-11-15 08:05:45,172 - NostalgiaForInfinityNextGen - INFO - pandas_ta successfully imported
2021-11-15 08:05:45,368 - freqtrade.resolvers.iresolver - INFO - Using resolved strategy NostalgiaForInfinityNextGen from '/freqtrade/user_data/strategies/NostalgiaForInfinityNextGen.py'...
2021-11-15 08:05:45,373 - freqtrade.strategy.hyper - INFO - Found no parameter file.
2021-11-15 08:05:45,384 - freqtrade.strategy.hyper - INFO - No params for protection found, using default values.
2021-11-15 08:05:45,389 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'timeframe' with value in config file: 15m.
2021-11-15 08:05:45,392 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'stake_currency' with value in config file: USDT.
2021-11-15 08:05:45,395 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'stake_amount' with value in config file: unlimited.
2021-11-15 08:05:45,402 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'unfilledtimeout' with value in config file: {'buy': 10, 'sell': 30, 'unit': 'minutes'}.
2021-11-15 08:05:45,403 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using minimal_roi: {'0': 100.0}
2021-11-15 08:05:45,405 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using timeframe: 15m
2021-11-15 08:05:45,406 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stoploss: -0.5
2021-11-15 08:05:45,407 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop: False
2021-11-15 08:05:45,409 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop_positive: 0.01
2021-11-15 08:05:45,410 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop_positive_offset: 0.03
2021-11-15 08:05:45,421 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_only_offset_is_reached: True
2021-11-15 08:05:45,421 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using use_custom_stoploss: False
2021-11-15 08:05:45,422 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using process_only_new_candles: True
2021-11-15 08:05:45,422 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using order_types: {'buy': 'limit', 'sell': 'limit', 'trailing_stop_loss': 'limit', 'stoploss': 'limit', 'stoploss_on_exchange': False, 'stoploss_on_exchange_interval': 60, 'stoploss_on_exchange_limit_ratio': 0.99}
2021-11-15 08:05:45,422 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using order_time_in_force: {'buy': 'gtc', 'sell': 'gtc'}
2021-11-15 08:05:45,423 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stake_currency: USDT
2021-11-15 08:05:45,423 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stake_amount: unlimited
2021-11-15 08:05:45,423 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using protections: []
2021-11-15 08:05:45,423 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using startup_candle_count: 480
2021-11-15 08:05:45,424 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using unfilledtimeout: {'buy': 10, 'sell': 30, 'unit': 'minutes'}
2021-11-15 08:05:45,424 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using use_sell_signal: True
2021-11-15 08:05:45,424 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using sell_profit_only: False
2021-11-15 08:05:45,425 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using ignore_roi_if_buy_signal: True
2021-11-15 08:05:45,425 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using sell_profit_offset: 0.0
2021-11-15 08:05:45,425 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using disable_dataframe_checks: False
2021-11-15 08:05:45,425 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using ignore_buying_expired_candle_after: 0
2021-11-15 08:05:45,426 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2021-11-15 08:05:45,505 - freqtrade.resolvers.iresolver - INFO - Using resolved pairlist StaticPairList from '/freqtrade/freqtrade/plugins/pairlist/StaticPairList.py'...
2021-11-15 08:05:45,608 - freqtrade.resolvers.iresolver - INFO - Using resolved pairlist AgeFilter from '/freqtrade/freqtrade/plugins/pairlist/AgeFilter.py'...
2021-11-15 08:06:14,311 - freqtrade.resolvers.iresolver - INFO - Using resolved hyperoptloss SharpeHyperOptLoss from '/freqtrade/freqtrade/optimize/hyperopt_loss_sharpe.py'...
2021-11-15 08:06:14,323 - freqtrade.optimize.hyperopt - INFO - Using optimizer random state: 53879
2021-11-15 08:06:14,326 - freqtrade - ERROR - The 'buy' space is included into the hyperoptimization but no parameter for this space was not found in your Strategy. Please make sure to have parameters for this space enabled for optimization or remove the 'buy' space from hyperoptimization.
nakapola commented 2 years ago

NFI intentionally remove hyperops param some months ago as you can see this error 2021-11-15 08:06:14,326 - freqtrade - ERROR - The 'buy' space is included into the hyperoptimization but no parameter for this space was not found in your Strategy. Please make sure to have parameters for this space enabled for optimization or remove the 'buy' space from hyperoptimization.

huytrvan commented 2 years ago

Thanks @abc112116, I didn't know that!

Does that mean we can't useedge positioning as well?

eSeR1805 commented 2 years ago

These problems come from lack of better understanding of freqtrade and not related to strategy itself

magick93 commented 2 years ago

Hello

NFI intentionally remove hyperops param some months

I'm curious to understand the background to this.

Thanks