search

scrtlabs / catalyst

An Algorithmic Trading Library for Crypto-Assets in Python
http://enigma.co
Apache License 2.0
2.49k stars 724 forks source link

Delisted coins cause crash if have open positions #530

Open apovall opened 5 years ago

apovall commented 5 years ago

Hi there,

Currently running latest 0.5.21 version of catalyst and libraries that come with that install (via pip).

Description of Issue

Currently running a simple momentum backtest (6 day lookback, 3 day refresh interval), on a universe of assets on Binance. Am testing with all available quote currencies (ETH, BTC, BNB, USDT), running each as a separate instance / separate namespace (I am NOT combining all quote pairs (e.g. ada_eth, ada_btc etc) into a single universe).

Everything works fine, but found that holding any assets over the period during which they are delisted from the exchange causes the Portofolio Value to become 'nan'. (E.g. buy an asset on say 2018-10-19 (like icn, chat, trig), it becomes delisted while you're holding, then you can't sell it and PV breaks).

I'm guessing it's best that these assets which become delisted are included (and back tested with) to avoid survivorship bias, but how is it best to actually deal with them in the context of a trading / backtesting algorithm? Having PV going to 'nan' (which it seems is further related to 'returns' going to nan as well) breaks all the back testing metrics.

Here's the full error message I get:

RuntimeWarning Traceback (most recent call last)

in 330 quote_currency='btc', 331 start=pd.to_datetime('2016-01-01', utc=True), --> 332 end=pd.to_datetime('2019-01-01', utc=True)) 333 334 tempDF = tempDF.append(output) ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/utils/run_algo.py in run_algorithm(initialize, capital_base, start, end, handle_data, before_trading_start, analyze, data_frequency, data, bundle, bundle_timestamp, default_extension, extensions, strict_extensions, environ, live, remote, mail, exchange_name, quote_currency, algo_namespace, live_graph, analyze_live, simulate_orders, auth_aliases, stats_output, output) 641 simulate_orders=simulate_orders, 642 auth_aliases=auth_aliases, --> 643 stats_output=stats_output 644 ) ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/utils/run_algo.py in _run(handle_data, initialize, before_trading_start, analyze, algofile, algotext, defines, data_frequency, capital_base, data, bundle, bundle_timestamp, start, end, output, print_algo, local_namespace, environ, live, exchange, algo_namespace, quote_currency, live_graph, analyze_live, simulate_orders, auth_aliases, stats_output) 359 ).run( 360 data, --> 361 overwrite_sim_params=False, 362 ) 363 ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/exchange/exchange_algorithm.py in run(self, data, overwrite_sim_params) 401 def run(self, data=None, overwrite_sim_params=True): 402 perf = super(ExchangeTradingAlgorithmBacktest, self).run( --> 403 data, overwrite_sim_params 404 ) 405 # Rebuilding the stats to support minute data ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/exchange/exchange_algorithm.py in run(self, data, overwrite_sim_params) 358 data.attempts = self.attempts 359 return super(ExchangeTradingAlgorithmBase, self).run( --> 360 data, overwrite_sim_params 361 ) 362 ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/algorithm.py in run(self, data, overwrite_sim_params) 723 try: 724 perfs = [] --> 725 for perf in self.get_generator(): 726 perfs.append(perf) 727 ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/gens/tradesimulation.py in transform(self) 245 yield minute_msg 246 --> 247 risk_message = algo.perf_tracker.handle_simulation_end() 248 yield risk_message 249 ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/performance/tracker.py in handle_simulation_end(self) 468 algorithm_leverages=acl, 469 trading_calendar=self.trading_calendar, --> 470 treasury_curves=self.treasury_curves, 471 ) 472 ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/risk/report.py in __init__(self, algorithm_returns, sim_params, trading_calendar, treasury_curves, benchmark_returns, algorithm_leverages) 96 97 self.month_periods = self.periods_in_range( ---> 98 1, start_session, end_session 99 ) 100 self.three_month_periods = self.periods_in_range( ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/risk/report.py in periods_in_range(self, months_per, start_session, end_session) 155 trading_calendar=self.trading_calendar, 156 treasury_curves=self.treasury_curves, --> 157 algorithm_leverages=self.algorithm_leverages, 158 ) 159 ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/risk/period.py in __init__(self, start_session, end_session, returns, trading_calendar, treasury_curves, benchmark_returns, algorithm_leverages) 79 self.algorithm_leverages = algorithm_leverages 80 ---> 81 self.calculate_metrics() 82 83 def calculate_metrics(self): ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/catalyst/finance/risk/period.py in calculate_metrics(self) 136 self.sharpe = 0.0 137 self.downside_risk = downside_risk( --> 138 self.algorithm_returns.values 139 ) 140 ~/.pyenv/versions/pythonbot3.1/lib/python3.6/site-packages/empyrical/stats.py in downside_risk(returns, required_return, period, annualization) 659 660 downside_diff = _adjust_returns(returns, required_return).copy() --> 661 mask = downside_diff > 0 662 downside_diff[mask] = 0.0 663 squares = np.square(downside_diff) RuntimeWarning: invalid value encountered in greater ---//--- Here's the code I'm using for initialize and handle_data below: def initialize(context, **kwargs): context.CANDLE_SIZE = '1T' # T = 'minutely'. Therefore 1T = 1 minute. 15T = 15 minutes. #context.LOOK_BACK = int(60 * 24 * global_day) # num days. Used to determine look back period as multiple of base CANDLE_SIZE context.LOOK_BACK = int(global_day) # num days. Used to determine look back period as multiple of base CANDLE_SIZE context.REFRESH_INTERVAL = context.LOOK_BACK/2 # How often to check for changes in momentum - 1440 = 1 day.60 = 1 hour context.BUY_MOMENTUM_THRESH = 0 # Fractional Percentage. 1% context.SELL_MOMENTUM_THRESH = 0 context.active_pairs = 20 context.ORDER_SIZE_METHOD = 'dynamic' # else 'fixed' context.old_set = set() context.new_set = None context.PAIRS = get_universe(global_quote) context.TOTAL_PAIRS = len(context.PAIRS) context.counter = 0 # Minute / day counter context.set_commission(maker=0.001, taker=0.002) context.set_slippage(slippage=0.001) def handle_data(context, data): # Increment minute count to determine when to check momentum context.counter += 1 # Every refresih interval (60m, 120m, etc), do checks if context.counter >= context.REFRESH_INTERVAL: # Reset minute count context.counter = 0 momentum_df = pd.DataFrame({'pair': [], 'momentum': [], 'position': []}) # Get momentum for pairs for pair in context.PAIRS: # Get price data from _x_ lookback ago for each pair in universe current_asset = symbol(pair) # Try to get pair. If not active (no historic or current prices), # or delisted (historic but no current price), will pass as exception, otherwise continue try: # Get current price data for each pair in universe current = data.current(current_asset, fields=['close']) # Get historic data for pair in universe historic_close = data.history(current_asset, fields=['close'], bar_count=context.LOOK_BACK, frequency='1T') # If it's been deslited (either current or historic price returns as none) #if (current['close'] is not None) or (historic_close.head(1).close.item() is not None): # Check price momentum (basic positive / not positive) and assign to DataFrame momentum_df = momentum_df.append({'pair': pair, 'momentum': (current['close'] - historic_close.head(1).close.item()) / historic_close.head(1).close.item(), 'position': context.portfolio.positions[current_asset].amount}, ignore_index=True) except Exception as error: pass # Get all pairs with positive momentum top_list = momentum_df.loc[(momentum_df.momentum > context.BUY_MOMENTUM_THRESH)].copy() # Order the list, descending from highest to lowest momentum top_list.sort_values(by='momentum', axis=0, inplace=True, ascending=False) if context.ORDER_SIZE_METHOD == 'dynamic': # Buy all pairs with positive momentum buy_list = top_list.copy() context.active_pairs = len(buy_list) elif context.ORDER_SIZE_METHOD == 'fixed': # Buy only pairs with positive momentum to a maximum number of context.active_pairs buy_list = top_list.head(context.active_pairs) # Create sell list with pairs not in buy list using set difference sell_list = momentum_df.drop(labels=buy_list.index) sell_list = sell_list.loc[sell_list.position > 0].pair # Keep list: Currently being traded, with position size greater than 0 and momentum above sell threshold still #keep_list = momentum_df.loc[(momentum_df.position > 0) & (momentum_df.momentum > context.SELL_MOMENTUM_THRESH)].pair # Buy list: Currently not being traded, but have momentum greater than threshold #buy_list = momentum_df.loc[(momentum_df.momentum >= context.BUY_MOMENTUM_THRESH) & (momentum_df.position == 0)].pair #combined_active_list = list(keep_list) + list(buy_list) combined_active_list = list(buy_list.pair) # Sell list: Currently being traded, but momentum has dropped below threshold to keep #sell_list = list(momentum_df.loc[(momentum_df.position > 0) & (momentum_df.momentum <= context.SELL_MOMENTUM_THRESH)].pair) # Sell everything in sell_list first if not len(sell_list) == 0: for sell_pair in sell_list: # If possible to trade pair if data.can_trade(symbol(sell_pair)) and not get_open_orders(symbol(sell_pair)): # Determine pair to sell, convert to usable symbol sell_target = symbol(sell_pair) # Set order percentage to 0, equivalent of a sell order_target_percent(sell_target, 0) # Get momentum to display for logging purposes momentum = momentum_df.loc[momentum_df.pair == sell_pair].momentum.item() # Log sell action log.info('{}: {}: Sell - momentum: {} - PV: {} - Buy: {} Sell: {}'.format( data.current_dt, sell_target, round(momentum, 5), round(context.portfolio.portfolio_value, 5), len(buy_list), len(sell_list))) # Sell # Buy everything (update position size) in combined_active_list second if not len(combined_active_list) == 0: for buy_pair in combined_active_list: # If possible to trade pair if data.can_trade(symbol(buy_pair)) and not get_open_orders(symbol(buy_pair)): # Determine pair to sell, convert to usable symbol buy_target = symbol(buy_pair) # Take a position of the size ORDER_SIZE for new pair order_target_percent(buy_target, 1/context.active_pairs) # Get momentum to display for logging purposes momentum = momentum_df.loc[momentum_df.pair == buy_pair].momentum.item() # Log buy action if not len(buy_list) == 0: log.info('{}: {}: Buy - momentum: {} - PV: {} - Buy: {} Sell: {}'.format( data.current_dt, buy_target, round(momentum, 5), round(context.portfolio.portfolio_value, 5), len(buy_list), len(sell_list))) # Buy record(active_pairs=context.active_pairs, buy_list=buy_list, sell_list=sell_list) Is this a bug, or is there some best practice way to deal with this? Cheers, let me know if you need any other info.
avolution commented 5 years ago

For Bitfinex you can realize that with if(pair['info']['expiration'] != 'NA') -> kick pair out of the universe

When you rebalance in short intervals.

But when I remember right, binance does not have a field like this.