Notebook: Analyzing strategy fails with errors

[gaardiolor]

Describe your environment

• Operating system: Windows
• Python Version: 3.10.4
• CCXT version: 1.92.30
• Freqtrade Version: 2022.7

Describe the problem:

I am trying to analyze my strategy, following https://www.freqtrade.io/en/stable/strategy_analysis_example/ . However, I'm getting an error.

Steps to reproduce:

My code:

from pathlib import Path
from freqtrade.configuration import Configuration

# Initialize empty configuration object
config = Configuration.from_files([])

# Define some constants
config["timeframe"] = "5m"
config["user_data_dir"] = Path("Y:/ft.strat.gateio.t4/")
config['strategy_path'] = Path("Y:/ft.strat.gateio.t4/strategies")
config['trading_mode'] = 'futures'

# Name of the strategy class
config["strategy"] = "strat"
config['candle_type_def'] = 'futures'

# Location of the data
data_location = Path(config['user_data_dir'], 'data', 'gateio')

# Pair to analyze - Only use one pair here
pair = "BTC/USDT:USDT"

from freqtrade.data.history import load_pair_history
from freqtrade.enums import CandleType

candles = load_pair_history(datadir=data_location,
                            timeframe=config["timeframe"],
                            pair=pair,
                            data_format = "json",
                            candle_type=config['candle_type_def'],
                            )

# Confirm success
print("Loaded " + str(len(candles)) + f" rows of data for {pair} from {data_location}")
candles.head()

Candles are printed, so this works.

# Load strategy using values set above
from freqtrade.resolvers import StrategyResolver
from freqtrade.data.dataprovider import DataProvider
strategy = StrategyResolver.load_strategy(config)
strategy.dp = DataProvider(config, None, None)

# Generate buy/sell signals using strategy
df = strategy.analyze_ticker(candles, {'pair': pair})
df.tail()

This gives an error:

---------------------------------------------------------------------------
OperationalException                      Traceback (most recent call last)
Input In [15], in <cell line: 2>()
      1 # Generate buy/sell signals using strategy
----> 2 df = strategy.analyze_ticker(candles, {'pair': pair})
      3 df.tail()

File ~\anaconda3\envs\ft_py310\lib\site-packages\freqtrade\strategy\interface.py:650, in IStrategy.analyze_ticker(self, dataframe, metadata)
    642 """
    643 Parses the given candle (OHLCV) data and returns a populated DataFrame
    644 add several TA indicators and entry order signal to it
   (...)
    647 :return: DataFrame of candle (OHLCV) data with indicator data and signals added
    648 """
    649 logger.debug("TA Analysis Launched")
--> 650 dataframe = self.advise_indicators(dataframe, metadata)
    651 dataframe = self.advise_entry(dataframe, metadata)
    652 dataframe = self.advise_exit(dataframe, metadata)

File ~\anaconda3\envs\ft_py310\lib\site-packages\freqtrade\strategy\interface.py:1144, in IStrategy.advise_indicators(self, dataframe, metadata)
   1142 # call populate_indicators_Nm() which were tagged with @informative decorator.
   1143 for inf_data, populate_fn in self._ft_informative:
-> 1144     dataframe = _create_and_merge_informative_pair(
   1145         self, dataframe, metadata, inf_data, populate_fn)
   1147 return self.populate_indicators(dataframe, metadata)

File ~\anaconda3\envs\ft_py310\lib\site-packages\freqtrade\strategy\informative_decorator.py:93, in _create_and_merge_informative_pair(strategy, dataframe, metadata, inf_data, populate_indicators)
     89 else:
     90     # Not specifying an asset will define informative dataframe for current pair.
     91     asset = metadata['pair']
---> 93 market = strategy.dp.market(asset)
     94 if market is None:
     95     raise OperationalException(f'Market {asset} is not available.')

File ~\anaconda3\envs\ft_py310\lib\site-packages\freqtrade\data\dataprovider.py:241, in DataProvider.market(self, pair)
    235 """
    236 Return market data for the pair
    237 :param pair: Pair to get the data for
    238 :return: Market data dict from ccxt or None if market info is not available for the pair
    239 """
    240 if self._exchange is None:
--> 241     raise OperationalException(NO_EXCHANGE_EXCEPTION)
    242 return self._exchange.markets.get(pair)

OperationalException: Exchange is not available to DataProvider.

I have also tried (found on discord):

config['exchange']['name'] = 'gateio'
from freqtrade.exchange import Exchange
strategy = StrategyResolver.load_strategy(config)
strategy.dp = DataProvider(config, Exchange(config), None)
# Generate buy/sell signals using strategy
df = strategy.analyze_ticker(candles, {'pair': pair})
df.tail()

This gives another error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Input In [24], in <cell line: 4>()
      2 from freqtrade.exchange import Exchange
      3 strategy = StrategyResolver.load_strategy(config)
----> 4 strategy.dp = DataProvider(config, Exchange(config), None)
      5 # Generate buy/sell signals using strategy
      6 df = strategy.analyze_ticker(candles, {'pair': pair})

File ~\anaconda3\envs\ft_py310\lib\site-packages\freqtrade\exchange\exchange.py:181, in Exchange.__init__(self, config, validate, load_leverage_tiers)
    177 logger.info(f'Using Exchange "{self.name}"')
    179 if validate:
    180     # Initial markets load
--> 181     self._load_markets()
    182     self.validate_config(config)
    183     self.required_candle_call_count = self.validate_required_startup_candles(
    184         config.get('startup_candle_count', 0), config.get('timeframe', ''))

File ~\anaconda3\envs\ft_py310\lib\site-packages\freqtrade\exchange\exchange.py:475, in Exchange._load_markets(self)
    473 try:
    474     self._markets = self._api.load_markets()
--> 475     self._load_async_markets()
    476     self._last_markets_refresh = arrow.utcnow().int_timestamp
    477     if self._ft_has['needs_trading_fees']:

File ~\anaconda3\envs\ft_py310\lib\site-packages\freqtrade\exchange\exchange.py:464, in Exchange._load_async_markets(self, reload)
    462 try:
    463     if self._api_async:
--> 464         self.loop.run_until_complete(
    465             self._api_async.load_markets(reload=reload))
    467 except (asyncio.TimeoutError, ccxt.BaseError) as e:
    468     logger.warning('Could not load async markets. Reason: %s', e)

File ~\anaconda3\envs\ft_py310\lib\asyncio\base_events.py:622, in BaseEventLoop.run_until_complete(self, future)
    611 """Run until the Future is done.
    612 
    613 If the argument is a coroutine, it is wrapped in a Task.
   (...)
    619 Return the Future's result, or raise its exception.
    620 """
    621 self._check_closed()
--> 622 self._check_running()
    624 new_task = not futures.isfuture(future)
    625 future = tasks.ensure_future(future, loop=self)

File ~\anaconda3\envs\ft_py310\lib\asyncio\base_events.py:584, in BaseEventLoop._check_running(self)
    582     raise RuntimeError('This event loop is already running')
    583 if events._get_running_loop() is not None:
--> 584     raise RuntimeError(
    585         'Cannot run the event loop while another loop is running')

RuntimeError: Cannot run the event loop while another loop is running

Observed Results:

• What happened? See above

• What did you expect to happen? No error.

Relevant code exceptions or logs

See above. Any tips.. ? Thanks!

[xmatthias]

Using the exchange in an interactive environment is unfortunately not supported due to limitations in jupyter notebooks (related to how asyncio works in combination with notebooks).

This will implicitly (and unfortunately) also limit which strategies can be used in the notebook environment.

While there's workarounds - i'd not really trust these to "really" work reliably - as they do some shady patching of python internals.

I'm working on a long-term solution to that (not explicitly for this - but it'll solve this implicitly) - but that'll still take a some time to be fully ready.

[froggleston]

Using notebooks works fine if you fulfil all the objects needed, including the workaround for asyncio:

...
## somewhere after the main imports but before the main notebook code
import nest_asyncio
nest_asyncio.apply()

...

ft_config['strategy'] = strat_name
ft_config['timeframe'] = timeframe
ft_config['timeframe_detail'] = timeframe_detail ## if required
ft_config['datadir'] = data_location
ft_exchange = ExchangeResolver.load_exchange(ft_config['exchange']['name'], config=ft_config, validate=True)
ft_pairlists = PairListManager(ft_exchange, ft_config)
ft_dataprovider = DataProvider(ft_config, ft_exchange, ft_pairlists)

# Load strategy using values set above
strategy = StrategyResolver.load_strategy(ft_config)
strategy.dp = ft_dataprovider

Taken from my notebook example at https://github.com/froggleston/cryptofrog-strategies/blob/informatives/live_plotting.ipynb which is old, but the main functionality should still be relevant.

Edit: However xmatthias is correct - this isn't a supported environment so YMMV.

[gaardiolor]

Thanks! I'll test this with @froggleston 's tips in Jupyter, and outside Jupyter. I'll update this issue with my findings.

[xmatthias]

To initialize the exchange, nest_asyncio.apply() will suffice (i think you've had most of the other code more or less correct). This is however patching python internals - and is not a supported method to use this.

Without looking at that code, and carefully reviewing what exactly it's patching, it could also cause misalignment with instances where it's used without this patch.

You're obviously free to use this workaround - just be warned that it's untested, and may lead to differences elsewhere.

[gaardiolor]

Yes, it's working now. The missing nest_asyncio.apply() was the main thing, after which I was able to get it to work. Thanks!

[xmatthias]

as said above - pay close attention to results - and in case of differences to other modes, assume it could maybe also be caused by this.