can't run benchmark strategy when using a config file without entry and exit pricing for futures

[vaidab]

Running a backtest with a config file that doesn't have entry and exit pricing errors with an 'exit_pricing' message.

Run benchmark strategy -> results = notebook_helper.backtest_all([bench_config], parallel, proportion_cores_parallel, data_location=data_location, data_format=data_format, trading_mode=trading_mode, timeframe_detail=timeframe_detail) notebook_helper.py ->

def backtest_one(config, config_i, pairlist_fmt, pair_count, timerange):
...
    config['pairs'] = config['exchange']['pair_whitelist']
    backtesting = Backtesting(config)
    data, timerange = backtesting.load_bt_data()
...

The output is:

Creating user_data/backtest_results/Erumperator-results_20210801-20211031-4_1000_100.pkl {'strategy': 'Erumperator', 'pair_count': [68], 'config': ['user_data/config/pairlist-static-binance-usdt.json', {'max_open_trades': 4, 'dry_run_wallet': 1000, 'stake_amount': 100, 'stake_currency': 'USDT', 'exchange': {'name': 'binance'}, 'export': 'signals', 'datadir': 'user_data/data/binance', 'trading_mode': 'futures', 'margin_mode': 'isolated', 'dataformat_ohlcv': 'hdf5', 'stoploss': -1, 'minimal_roi': {'0': 100}}], 'timeranges': ['20210801-20210831', '20210901-20210930', '20211001-20211031']} user_data/data/binance hdf5 futures

0% 0/3 [00:00<?, ?it/s] 'exit_pricing'

The 'exit_pricing' is an exception/output from the Backtesting() function that is imported. The code doesn't arrive to the next line, data, timerange = backtesting.load_bt_data()

Reason: no entry_pricing and exit_pricing in Configuration -> if_short -> config = {}

This can't be reproduced if your config file contains these values, but if it doesn't (e.g. you're using a pairlist file only) then it gives that output.

This will be fixed in my pull request.

[froggleston]

I'm not sure that this is a viable fix. The user needs to supply their own exit_pricing config and you cannot run backtesting simply with a pairlist file in any case, as backtesting from the command line would fail here also.

[vaidab]

I'm running it with only the pairlist, that's why I've added those entries. The fix is more for the 'entry_pricing' error message that a user would encounter if they'd use a pairlist instead of a full config.

Alternatively, we could also add a second config file (a pairlist file) in the beginning and allow multiple config files, but this also works and prevents that issue.

[froggleston]

You are running with only the pairlist, yes, but others are more than likely to not do this. The assumption is that they use a full config because that's what backtesting from the command line expects. I'd prefer to default to the most common use case, and let people split things out if they wanted to.

[vaidab]

Agreed to let the most common use case be the default. Since I'm using the default values for entry and exit pricing, it's a convenience to leave them there for people using the pairlists and it doesn't inconvenience users using a full config file, right? It's like leaving a commented freqtrade dir for docker users while keeping the default dir for non-docker, difference is that they can both work with one dir.

[froggleston]

My point is where does it end? Do we add every config option as an option or do we try and keep the most common ones available, and the others as the ones a user must set as they would be command line backtesting? I'd go for the latter.

[vaidab]

I agree with you regarding the thinking. Here, I'd say there are 2 scenarios: full config and pairlist file. The user wouldn't use another format (like a secret config with the passwords or incomplete config files), but they would use any of these 2 options. So we're catering to category B and we shouldn't go further than that. But we should do it for cat B as they would receive a weird error and they might lack the skill or understanding of what is happening there (I also took my time debugging this issue and didn't get that it was because of my conf file setup).