search

freqtrade / freqtrade

Free, open source crypto trading bot
https://www.freqtrade.io
GNU General Public License v3.0
28.45k stars 6.1k forks source link

Why my hyperopt results differ from backtest? #7131

Closed fridary closed 2 years ago

fridary commented 2 years ago

I noticed weird thing my hyperopt results differ from backtest, the same I see on older versions.

{
    "max_open_trades": -1,
    "stake_amount": 1000,
    "stake_currency": "USDT",
    "available_capital": 25000,
    "dry_run": true,
    "dry_run_wallet": 25000,
    "trading_mode": "futures",
    "margin_mode": "isolated",
    ...
    "exchange": {
        "name": "binance",
        ...
    },
    ...

python -m freqtrade download-data -c 'config.json' -t 5m --days 100 -p SOL/USDT

$ python -m freqtrade hyperopt -c 'config.json' --disable-param-export --print-all --hyperopt-loss SharpeHyperOptLoss -j -1 -e 30 --min-trades 1 --spaces buy -s adaptive_bug -p SOL/USDT --timeframe 5m --timerange 20220501-20220601 --random-state 45735
2022-07-25 18:24:35,965 - freqtrade.configuration.load_config - INFO - Using config: user_data/config.json ...
2022-07-25 18:24:35,966 - freqtrade.configuration.load_config - INFO - Using config: user_data/backtest.json ...
2022-07-25 18:24:35,967 - freqtrade.loggers - INFO - Verbosity set to 0
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Parameter -i/--timeframe detected ... Using timeframe: 5m ...
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Using max_open_trades: -1 ...
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Parameter --timerange detected: 20220501-20220601 ...
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Using user-data directory: /python/freqtrade-clean/user_data ...
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Using data directory: /data_crypto ...
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Overriding timeframe with Command line argument
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Parameter --disableparamexport detected: True ...
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Parameter --epochs detected ... Will run Hyperopt with for 30 epochs ...
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Parameter -s/--spaces detected: ['buy']
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Parameter --print-all detected ...
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Parameter -j/--job-workers detected: -1
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Parameter --random-state detected: 45735
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Parameter --min-trades detected: 1
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Using Hyperopt loss class name: SharpeHyperOptLoss
2022-07-25 18:24:35,967 - freqtrade.configuration.configuration - INFO - Using pairs ['SOL/USDT']
2022-07-25 18:24:35,967 - freqtrade.configuration.check_exchange - INFO - Checking exchange...
2022-07-25 18:24:35,973 - freqtrade.configuration.check_exchange - INFO - Exchange "binance" is officially supported by the Freqtrade development team.
2022-07-25 18:24:35,973 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2022-07-25 18:24:35,975 - freqtrade.commands.optimize_commands - INFO - Starting freqtrade in Hyperopt mode
2022-07-25 18:24:35,975 - freqtrade.exchange.exchange - INFO - Instance is running with dry_run enabled
2022-07-25 18:24:35,976 - freqtrade.exchange.exchange - INFO - Using CCXT 1.82.42
2022-07-25 18:24:35,976 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'future'}}
2022-07-25 18:24:35,981 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'future'}}
2022-07-25 18:24:35,986 - freqtrade.exchange.exchange - INFO - Using Exchange "Binance"
2022-07-25 18:24:36,877 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Binance'...
2022-07-25 18:24:36,962 - freqtrade.resolvers.iresolver - INFO - Using resolved strategy adaptive_bug from '/freq-strategies/adaptive_bug.py'...
2022-07-25 18:24:36,962 - freqtrade.strategy.hyper - INFO - Found no parameter file.
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'timeframe' with value in config file: 5m.
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'stake_currency' with value in config file: USDT.
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'stake_amount' with value in config file: 1000.
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'unfilledtimeout' with value in config file: {'entry': 10, 'exit': 10, 'exit_timeout_count': 0, 'unit': 'minutes'}.
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using minimal_roi: {'0': 0.02}
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using timeframe: 5m
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stoploss: -0.045
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop: False
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop_positive_offset: 0.0
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_only_offset_is_reached: False
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using use_custom_stoploss: False
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using process_only_new_candles: True
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using order_types: {'entry': 'limit', 'exit': 'limit', 'stoploss': 'limit', 'stoploss_on_exchange': False, 'stoploss_on_exchange_interval': 60}
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using order_time_in_force: {'entry': 'gtc', 'exit': 'gtc'}
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stake_currency: USDT
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stake_amount: 1000
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using protections: []
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using startup_candle_count: 100
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using unfilledtimeout: {'entry': 10, 'exit': 10, 'exit_timeout_count': 0, 'unit': 'minutes'}
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using use_exit_signal: True
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using exit_profit_only: False
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using ignore_roi_if_entry_signal: False
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using exit_profit_offset: 0.0
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using disable_dataframe_checks: False
2022-07-25 18:24:36,962 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using ignore_buying_expired_candle_after: 0
2022-07-25 18:24:36,963 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using position_adjustment_enable: False
2022-07-25 18:24:36,963 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using max_entry_position_adjustment: -1
2022-07-25 18:24:36,963 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2022-07-25 18:24:36,965 - freqtrade.resolvers.iresolver - INFO - Using resolved pairlist StaticPairList from '/freqtrade/plugins/pairlist/StaticPairList.py'...
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - No params for buy found, using default values.
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - Strategy Parameter(default): buy_cti = 14
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - Strategy Parameter(default): buy_cti_cond = 0.715
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - Strategy Parameter(default): buy_kama = 80
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - Strategy Parameter(default): buy_mama = 0.5
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - Strategy Parameter(default): buy_mama_diff = 0.025
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - Strategy Parameter(default): buy_mama_factor = 1.0
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - Strategy Parameter(default): buy_rsi = 84
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - Strategy Parameter(default): buy_rsi_cond = 60
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - No params for sell found, using default values.
2022-07-25 18:24:36,967 - freqtrade.strategy.hyper - INFO - No params for protection found, using default values.
2022-07-25 18:24:36,969 - freqtrade.resolvers.iresolver - INFO - Using resolved hyperoptloss SharpeHyperOptLoss from '/python/freqtrade-clean/freqtrade/optimize/hyperopt_loss/hyperopt_loss_sharpe.py'...
2022-07-25 18:24:36,969 - freqtrade.optimize.hyperopt - INFO - Removing `/python/freqtrade-clean/user_data/hyperopt_results/hyperopt_tickerdata.pkl`.
2022-07-25 18:24:36,969 - freqtrade.optimize.hyperopt - INFO - Using optimizer random state: 45735
2022-07-25 18:24:36,971 - freqtrade.data.history.history_utils - INFO - Using indicator startup period: 100 ...
2022-07-25 18:24:37,081 - freqtrade.optimize.backtesting - INFO - Loading data from 2022-04-30 15:40:00 up to 2022-06-01 00:00:00 (31 days).
2022-07-25 18:24:37,095 - freqtrade.optimize.hyperopt - INFO - Dataload complete. Calculating indicators
2022-07-25 18:24:37,148 - freqtrade.optimize.hyperopt - INFO - Hyperopting with data from 2022-05-01 00:00:00 up to 2022-06-01 00:00:00 (31 days)..
2022-07-25 18:24:37,149 - freqtrade.exchange.exchange - INFO - Closing async ccxt session.
2022-07-25 18:24:37,150 - freqtrade.optimize.hyperopt - INFO - Found 8 CPU cores. Let's make them scream!
2022-07-25 18:24:37,150 - freqtrade.optimize.hyperopt - INFO - Number of parallel jobs set as: -1
2022-07-25 18:24:37,150 - freqtrade.optimize.hyperopt - INFO - Using estimator ET.
2022-07-25 18:24:37,156 - freqtrade.optimize.hyperopt - INFO - Effective number of parallel workers used: 8
+--------+---------+----------+------------------+--------------+-------------------------------+-----------------+-------------+-------------------------------+
|   Best |   Epoch |   Trades |    Win Draw Loss |   Avg profit |                        Profit |    Avg duration |   Objective |           Max Drawdown (Acct) |
|--------+---------+----------+------------------+--------------+-------------------------------+-----------------+-------------+-------------------------------|
| * Best |    1/30 |       64 |     43    0   21 |       -0.14% |       -90.741 USDT   (-0.36%) | 0 days 01:06:00 |     2.46247 |       280.339 USDT    (1.12%) |
| * Best |    2/30 |       35 |     24    0   11 |       -0.06% |       -19.301 USDT   (-0.08%) | 0 days 00:05:00 |     0.74719 |       181.564 USDT    (0.72%) |                                                              
| *      |    3/30 |       86 |     56    0   30 |       -0.28% |      -238.987 USDT   (-0.96%) | 0 days 01:19:00 |     5.57399 |       497.979 USDT    (1.99%) |
| *      |    4/30 |       78 |     52    0   26 |       -0.18% |      -137.423 USDT   (-0.55%) | 0 days 01:24:00 |     3.52575 |       356.415 USDT    (1.42%) |
| *      |    5/30 |       80 |     52    0   28 |       -0.25% |      -198.496 USDT   (-0.79%) | 0 days 01:55:00 |     4.76226 |       376.805 USDT    (1.51%) |
| *      |    6/30 |       71 |     48    0   23 |       -0.11% |       -81.584 USDT   (-0.33%) | 0 days 01:08:00 |       2.357 |       280.579 USDT    (1.12%) |
| *      |    7/30 |       13 |      8    0    5 |       -0.51% |       -66.955 USDT   (-0.27%) | 0 days 00:01:00 |     1.42301 |       136.173 USDT    (0.55%) |                                                              
| *      |    8/30 |       29 |     19    0   10 |       -0.25% |       -73.910 USDT   (-0.30%) | 0 days 00:04:00 |     1.75309 |       181.564 USDT    (0.73%) |
| *      |    9/30 |       43 |     28    0   15 |       -0.28% |      -120.848 USDT   (-0.48%) | 0 days 00:09:00 |     2.81496 |       270.083 USDT    (1.08%) |
| * Best |   10/30 |       21 |     16    0    5 |        0.44% |        93.429 USDT    (0.37%) | 0 days 00:10:00 |    -1.83431 |        76.173 USDT    (0.30%) |                                                              
| *      |   11/30 |       84 |     52    0   32 |       -0.49% |      -409.825 USDT   (-1.64%) | 0 days 01:05:00 |     8.76533 |       528.716 USDT    (2.12%) |
| *      |   12/30 |        6 |      5    0    1 |        0.91% |        54.609 USDT    (0.22%) | 0 days 00:00:00 |    -1.30513 |        45.391 USDT    (0.18%) |
| *      |   13/30 |       10 |      7    0    3 |        0.04% |         3.827 USDT    (0.02%) | 0 days 00:00:00 |     0.02414 |        45.391 USDT    (0.18%) |
| * Best |   14/30 |       31 |     25    0    6 |        0.74% |       229.139 USDT    (0.92%) | 0 days 01:40:00 |    -5.10066 |        95.758 USDT    (0.38%) |
| *      |   15/30 |       50 |     32    0   18 |       -0.35% |      -176.870 USDT   (-0.71%) | 0 days 00:20:00 |     3.96345 |       360.865 USDT    (1.44%) |
| *      |   16/30 |       14 |      9    0    5 |       -0.34% |       -46.955 USDT   (-0.19%) | 0 days 00:02:00 |     1.06128 |        90.782 USDT    (0.36%) |                                                              
| *      |   17/30 |       21 |     17    0    4 |        0.76% |       158.604 USDT    (0.63%) | 0 days 00:31:00 |    -3.55407 |        76.173 USDT    (0.30%) |
| *      |   18/30 |       34 |     23    0   11 |       -0.12% |       -39.301 USDT   (-0.16%) | 0 days 00:05:00 |      1.1343 |       181.564 USDT    (0.72%) |                                                              
| *      |   19/30 |       71 |     45    0   26 |       -0.39% |      -277.696 USDT   (-1.11%) | 0 days 01:01:00 |     6.12366 |       467.294 USDT    (1.87%) |
| *      |   20/30 |       20 |     13    0    7 |       -0.29% |       -57.737 USDT   (-0.23%) | 0 days 00:04:00 |     1.33849 |       136.173 USDT    (0.55%) |
| *      |   21/30 |       94 |     62    0   32 |       -0.22% |      -209.944 USDT   (-0.84%) | 0 days 01:16:00 |     5.10893 |       508.761 USDT    (2.03%) |
| *      |   22/30 |       13 |      8    0    5 |       -0.51% |       -66.955 USDT   (-0.27%) | 0 days 00:01:00 |     1.42301 |       136.173 USDT    (0.55%) |
| *      |   23/30 |       14 |     11    0    3 |        0.60% |        83.827 USDT    (0.34%) | 0 days 00:02:00 |    -1.76459 |        90.782 USDT    (0.36%) |
| *      |   24/30 |       79 |     49    0   30 |       -0.48% |      -379.086 USDT   (-1.52%) | 0 days 01:05:00 |     8.12538 |       477.934 USDT    (1.91%) |                                                              
| *      |   25/30 |        5 |      5    0    0 |        2.00% |       100.000 USDT    (0.40%) | 0 days 00:00:00 |          20 |                            -- |
| *      |   26/30 |        9 |      7    0    2 |        0.55% |        49.218 USDT    (0.20%) | 0 days 00:00:00 |    -1.01372 |        45.391 USDT    (0.18%) |                                                              
| *      |   27/30 |       41 |     26    0   15 |       -0.39% |      -160.481 USDT   (-0.64%) | 0 days 00:10:00 |     3.54049 |       290.083 USDT    (1.16%) |
| *      |   28/30 |       77 |     48    0   29 |       -0.46% |      -353.478 USDT   (-1.41%) | 0 days 01:06:00 |     7.62063 |       432.543 USDT    (1.73%) |
| *      |   29/30 |       15 |      9    0    6 |       -0.62% |       -92.346 USDT   (-0.37%) | 0 days 00:03:00 |     1.92087 |       136.173 USDT    (0.55%) |
| *      |   30/30 |       13 |      8    0    5 |       -0.51% |       -66.955 USDT   (-0.27%) | 0 days 00:01:00 |     1.42301 |       136.173 USDT    (0.55%) |
 [Epoch 30 of 30 (100%)] ||                                                                                                                                                           | [Time:  0:00:22, Elapsed Time: 0:00:22]
2022-07-25 18:25:03,728 - freqtrade.optimize.hyperopt - INFO - 30 epochs saved to '/python/freqtrade-clean/user_data/hyperopt_results/strategy_adaptive_bug_2022-07-25_18-24-36.fthypt'.

Best result:

*   14/30:     31 trades. 25/0/6 Wins/Draws/Losses. Avg profit   0.74%. Median profit   2.00%. Total profit 229.13922280 USDT (   0.92%). Avg duration 1:40:00 min. Objective: -5.10066

    # Buy hyperspace params:
    buy_params = {
        "buy_cti": 53,
        "buy_cti_cond": 0.865,
        "buy_kama": 42,
        "buy_mama": 0.714,
        "buy_mama_diff": 0.018,
        "buy_mama_factor": 1.034,
        "buy_rsi": 51,
        "buy_rsi_cond": 66,
    }

    # ROI table:  # value loaded from strategy
    minimal_roi = {
        "0": 0.02
    }

    # Stoploss:
    stoploss = -0.045  # value loaded from strategy

    # Trailing stop:
    trailing_stop = False  # value loaded from strategy
    trailing_stop_positive = None  # value loaded from strategy
    trailing_stop_positive_offset = 0.0  # value loaded from strategy
    trailing_only_offset_is_reached = False  # value loaded from strategy

Now, I add buy_params to strategy:

class adaptive_bug(IStrategy):
    buy_params = {
        "buy_cti": 53,
        "buy_cti_cond": 0.865,
        "buy_kama": 42,
        "buy_mama": 0.714,
        "buy_mama_diff": 0.018,
        "buy_mama_factor": 1.034,
        "buy_rsi": 51,
        "buy_rsi_cond": 66,
    }

I got this:

$ python -m freqtrade backtesting -c 'config.json' -s adaptive_bug -p SOL/USDT --timeframe 5m --timerange 20220501-20220601 --cache none
2022-07-25 18:29:36,195 - freqtrade.configuration.load_config - INFO - Using config: user_data/config.json ...
2022-07-25 18:29:36,196 - freqtrade.configuration.load_config - INFO - Using config: user_data/backtest.json ...
2022-07-25 18:29:36,196 - freqtrade.loggers - INFO - Verbosity set to 0
2022-07-25 18:29:36,196 - freqtrade.configuration.configuration - INFO - Parameter -i/--timeframe detected ... Using timeframe: 5m ...
2022-07-25 18:29:36,196 - freqtrade.configuration.configuration - INFO - Using max_open_trades: -1 ...
2022-07-25 18:29:36,196 - freqtrade.configuration.configuration - INFO - Parameter --timerange detected: 20220501-20220601 ...
2022-07-25 18:29:36,196 - freqtrade.configuration.configuration - INFO - Using user-data directory: /python/freqtrade-clean/user_data ...
2022-07-25 18:29:36,197 - freqtrade.configuration.configuration - INFO - Using data directory: /Downloads/data_crypto ...
2022-07-25 18:29:36,197 - freqtrade.configuration.configuration - INFO - Overriding timeframe with Command line argument
2022-07-25 18:29:36,197 - freqtrade.configuration.configuration - INFO - Parameter --cache=none detected ...
2022-07-25 18:29:36,197 - freqtrade.configuration.configuration - INFO - Using pairs ['SOL/USDT']
2022-07-25 18:29:36,197 - freqtrade.configuration.check_exchange - INFO - Checking exchange...
2022-07-25 18:29:36,202 - freqtrade.configuration.check_exchange - INFO - Exchange "binance" is officially supported by the Freqtrade development team.
2022-07-25 18:29:36,202 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2022-07-25 18:29:36,203 - freqtrade.commands.optimize_commands - INFO - Starting freqtrade in Backtesting mode
2022-07-25 18:29:36,204 - freqtrade.exchange.exchange - INFO - Instance is running with dry_run enabled
2022-07-25 18:29:36,204 - freqtrade.exchange.exchange - INFO - Using CCXT 1.82.42
2022-07-25 18:29:36,204 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'future'}}
2022-07-25 18:29:36,208 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'future'}}
2022-07-25 18:29:36,213 - freqtrade.exchange.exchange - INFO - Using Exchange "Binance"
2022-07-25 18:29:37,655 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Binance'...
2022-07-25 18:29:37,740 - freqtrade.resolvers.iresolver - INFO - Using resolved strategy adaptive_bug from '/python/freq-strategies-d__done/adaptive_bug.py'...
2022-07-25 18:29:37,740 - freqtrade.strategy.hyper - INFO - Found no parameter file.
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'timeframe' with value in config file: 5m.
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'stake_currency' with value in config file: USDT.
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'stake_amount' with value in config file: 1000.
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Override strategy 'unfilledtimeout' with value in config file: {'entry': 10, 'exit': 10, 'exit_timeout_count': 0, 'unit': 'minutes'}.
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using minimal_roi: {'0': 0.02}
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using timeframe: 5m
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stoploss: -0.045
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop: False
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_stop_positive_offset: 0.0
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using trailing_only_offset_is_reached: False
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using use_custom_stoploss: False
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using process_only_new_candles: True
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using order_types: {'entry': 'limit', 'exit': 'limit', 'stoploss': 'limit', 'stoploss_on_exchange': False, 'stoploss_on_exchange_interval': 60}
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using order_time_in_force: {'entry': 'gtc', 'exit': 'gtc'}
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stake_currency: USDT
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using stake_amount: 1000
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using protections: []
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using startup_candle_count: 100
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using unfilledtimeout: {'entry': 10, 'exit': 10, 'exit_timeout_count': 0, 'unit': 'minutes'}
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using use_exit_signal: True
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using exit_profit_only: False
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using ignore_roi_if_entry_signal: False
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using exit_profit_offset: 0.0
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using disable_dataframe_checks: False
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using ignore_buying_expired_candle_after: 0
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using position_adjustment_enable: False
2022-07-25 18:29:37,740 - freqtrade.resolvers.strategy_resolver - INFO - Strategy using max_entry_position_adjustment: -1
2022-07-25 18:29:37,740 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2022-07-25 18:29:37,744 - freqtrade.resolvers.iresolver - INFO - Using resolved pairlist StaticPairList from '/python/freqtrade-clean/freqtrade/plugins/pairlist/StaticPairList.py'...
2022-07-25 18:29:37,745 - freqtrade.data.history.history_utils - INFO - Using indicator startup period: 100 ...
2022-07-25 18:29:37,886 - freqtrade.optimize.backtesting - INFO - Loading data from 2022-04-30 15:40:00 up to 2022-06-01 00:00:00 (31 days).
2022-07-25 18:29:37,901 - freqtrade.optimize.backtesting - INFO - Dataload complete. Calculating indicators
2022-07-25 18:29:37,901 - freqtrade.optimize.backtesting - INFO - Running backtesting for Strategy adaptive_bug
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - Strategy Parameter: buy_cti = 53
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - Strategy Parameter: buy_cti_cond = 0.865
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - Strategy Parameter: buy_kama = 42
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - Strategy Parameter: buy_mama = 0.714
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - Strategy Parameter: buy_mama_diff = 0.018
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - Strategy Parameter: buy_mama_factor = 1.034
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - Strategy Parameter: buy_rsi = 51
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - Strategy Parameter: buy_rsi_cond = 66
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - No params for sell found, using default values.
2022-07-25 18:29:37,901 - freqtrade.strategy.hyper - INFO - No params for protection found, using default values.
2022-07-25 18:29:37,969 - freqtrade.optimize.backtesting - INFO - Backtesting with data from 2022-05-01 00:00:00 up to 2022-06-01 00:00:00 (31 days).
2022-07-25 18:29:38,077 - freqtrade.misc - INFO - dumping json to "/python/freqtrade-clean/user_data/backtest_results/backtest-result-2022-07-25_18-29-38.meta.json"
2022-07-25 18:29:38,078 - freqtrade.misc - INFO - dumping json to "/python/freqtrade-clean/user_data/backtest_results/backtest-result-2022-07-25_18-29-38.json"
2022-07-25 18:29:38,078 - freqtrade.misc - INFO - dumping json to "/python/freqtrade-clean/user_data/backtest_results/.last_result.json"
Result for strategy adaptive_bug
========================================================== BACKTESTING REPORT =========================================================
|     Pair |   Buys |   Avg Profit % |   Cum Profit % |   Tot Profit USDT |   Tot Profit % |   Avg Duration |   Win  Draw  Loss  Win% |
|----------+--------+----------------+----------------+-------------------+----------------+----------------+-------------------------|
| SOL/USDT |     19 |          -0.41 |          -7.77 |           -77.737 |          -0.31 |        0:24:00 |    12     0     7  63.2 |
|    TOTAL |     19 |          -0.41 |          -7.77 |           -77.737 |          -0.31 |        0:24:00 |    12     0     7  63.2 |
========================================================= ENTER TAG STATS ==========================================================
|   TAG |   Buys |   Avg Profit % |   Cum Profit % |   Tot Profit USDT |   Tot Profit % |   Avg Duration |   Win  Draw  Loss  Win% |
|-------+--------+----------------+----------------+-------------------+----------------+----------------+-------------------------|
| TOTAL |     19 |          -0.41 |          -7.77 |           -77.737 |          -0.31 |        0:24:00 |    12     0     7  63.2 |
===================================================== EXIT REASON STATS =====================================================
|   Exit Reason |   Exits |   Win  Draws  Loss  Win% |   Avg Profit % |   Cum Profit % |   Tot Profit USDT |   Tot Profit % |
|---------------+---------+--------------------------+----------------+----------------+-------------------+----------------|
|           roi |      12 |     12     0     0   100 |           2    |          24    |           240     |          24    |
|     stop_loss |       7 |      0     0     7     0 |          -4.54 |         -31.77 |          -317.737 |         -31.77 |
====================================================== LEFT OPEN TRADES REPORT ======================================================
|   Pair |   Buys |   Avg Profit % |   Cum Profit % |   Tot Profit USDT |   Tot Profit % |   Avg Duration |   Win  Draw  Loss  Win% |
|--------+--------+----------------+----------------+-------------------+----------------+----------------+-------------------------|
|  TOTAL |      0 |           0.00 |           0.00 |             0.000 |           0.00 |           0:00 |     0     0     0     0 |
================== SUMMARY METRICS ==================
| Metric                      | Value               |
|-----------------------------+---------------------|
| Backtesting from            | 2022-05-01 00:00:00 |
| Backtesting to              | 2022-06-01 00:00:00 |
| Max open trades             | 1                   |
|                             |                     |
| Total/Daily Avg Trades      | 19 / 0.61           |
| Starting balance            | 25000 USDT          |
| Final balance               | 24922.263 USDT      |
| Absolute profit             | -77.737 USDT        |
| Total profit %              | -0.31%              |
| CAGR %                      | -3.60%              |
| Profit factor               | 0.76                |
| Trades per day              | 0.61                |
| Avg. daily profit %         | -0.01%              |
| Avg. stake amount           | 1000 USDT           |
| Total trade volume          | 19000 USDT          |
|                             |                     |
| Best Pair                   | SOL/USDT -7.77%     |
| Worst Pair                  | SOL/USDT -7.77%     |
| Best trade                  | SOL/USDT 2.00%      |
| Worst trade                 | SOL/USDT -4.54%     |
| Best day                    | 20 USDT             |
| Worst day                   | -81.564 USDT        |
| Days win/draw/lose          | 3 / 12 / 3          |
| Avg. Duration Winners       | 0:09:00             |
| Avg. Duration Loser         | 0:49:00             |
| Rejected Entry signals      | 0                   |
| Entry/Exit Timeouts         | 0 / 0               |
|                             |                     |
| Min balance                 | 24907.654 USDT      |
| Max balance                 | 25060 USDT          |
| Max % of account underwater | 0.61%               |
| Absolute Drawdown (Account) | 0.61%               |
| Absolute Drawdown           | 152.346 USDT        |
| Drawdown high               | 60 USDT             |
| Drawdown low                | -92.346 USDT        |
| Drawdown Start              | 2022-05-11 20:00:00 |
| Drawdown End                | 2022-05-12 07:15:00 |
| Market change               | -50.38%             |
=====================================================

Any ideas what's wrong? Moreover, if I add buy_params to strategy class and make hyperopt again with the same --random-state 45735, it will print another results.

from freqtrade.strategy.interface import IStrategy
from pandas import DataFrame
import pandas_ta as pta
import talib.abstract as ta
from freqtrade.strategy import (DecimalParameter, IStrategy, IntParameter)
from functools import reduce

class adaptive_bug(IStrategy):

    INTERFACE_VERSION = 3

    minimal_roi = {
        "0": 0.02
    }

    stoploss = -0.045
    trailing_stop = False
    timeframe = '5m'
    startup_candle_count: int = 100

    buy_cti = IntParameter(7, 70, default=14, space='buy', optimize=True, load=True)
    buy_cti_cond = DecimalParameter(0.500, 0.900, decimals=3, default=0.715, space='buy', optimize=True, load=True)
    buy_mama = DecimalParameter(0.15, 0.85, decimals=3, default=0.5, space='buy', optimize=True, load=True)
    buy_mama_factor = DecimalParameter(0.90, 1.10, decimals=3, default=1.0, space='buy', optimize=True, load=True)
    buy_mama_diff = DecimalParameter(0.010, 0.045, decimals=3, default=0.025, space='buy', optimize=True, load=True)
    buy_kama = IntParameter(7, 150, default=80, space='buy', optimize=True, load=True)
    buy_rsi = IntParameter(7, 100, default=84, space='buy', optimize=True, load=True)
    buy_rsi_cond = IntParameter(50, 70, default=60, space='buy', optimize=True, load=True)

    def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        dataframe['mama'], dataframe['fama'] = ta.MAMA(dataframe['close'], self.buy_mama.value, self.buy_mama.value * self.buy_mama_factor.value / 10)
        dataframe['mama_diff'] = ((dataframe['mama'] - dataframe['fama']) / dataframe['close'])
        dataframe['kama'] = ta.KAMA(dataframe['close'], self.buy_kama.value)
        dataframe['cti'] = pta.cti(dataframe["close"], length=self.buy_cti.value)
        dataframe['rsi'] = ta.RSI(dataframe, timeperiod=self.buy_rsi.value)
        return dataframe

    def populate_entry_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        conditions = []
        conditions.append(
            (
                (dataframe['kama'] > dataframe['fama']) &
                (dataframe['fama'] > dataframe['mama'] * 0.981) &
                (dataframe['mama_diff'] < -1 * self.buy_mama_diff.value) &
                (dataframe['cti'] < -1 * self.buy_cti_cond.value) &
                (dataframe['rsi'] < self.buy_rsi_cond.value)
            )
        )
        if conditions:
            dataframe.loc[
                reduce(lambda x, y: x & y, conditions),
                'enter_long'
            ] = 1
        return dataframe

    def populate_exit_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        return dataframe
xmatthias commented 2 years ago

Moreover, if I add buy_params to strategy class and make hyperopt again with the same --random-state 45735, it will print another results.

This is (kindof) expected - having buy_params does not mean the parameters won't be optimized. This is being controlled by the --space flag - or by the optimize flag on each parameter.

In general, it'll not be necessary to add buy_params to the strategy itself. Hyperopt (or hyperopt-show) will write a parameter file (strategyfilename.json) - which is placed alongside the strategy - and loaded from there.

Now the backtesting logs indicate that this file is not found - which suggests some sort of interaction from your side (?)

Aside from that - the "same random-state" makes me suspicious.

if you run backtesting multiple times (no modification) - is the result identical, or different each time?

I've seen something similar before (random signals for every call) - and the reason then was an edge-case in a ta-lib indicator, which was called with an unexpected value - causing the calculation to crash (in that case it was MACD - but i assume the same can happen with every other value/indicator, too).

fridary commented 2 years ago

..sec

xmatthias commented 2 years ago

After looking at your code a bit more, i realized that your populate_indicator() function is wrong.

using .value in populate_indicators() is wrong. populate_indicators is called exactly once before running hyperopt - so.value` will never be different. This is a supported approach though - but requires a different syntax / approach. You can find details and an example for this in this section of the documentation.

What this does then is that (for example) ta.KAMA(dataframe['close'], self.buy_kama.value) is called with the initial value. while the value itself changes afterwards - it's never recaluclated (remember, populate_indicators() is only called once) - so your values between hyperopt and backtesting differs - because inputs wre different

fridary commented 2 years ago

My bad, sorry, I found it's already and you got ahead of me to text. Thank you anyway very much.