Develop Branch - NotEnoughCashError due to error in balance fetch

[zackgow]

I am currently working on the Develop branch.

I was Live Trading for ~14 hours, and then received a NotEnoughCashError. I was not trying to place an order when the error occurred. My cash and positions were being correctly tracked by Catalyst. I was trading ETH/USDT, with ~96 USDT in my account.

It seems that the error may be due to this function:

def _check_low_balance(self, currency, balances, amount, open_orders=None):
        free = balances[currency]['free'] if currency in balances else 0.0

It looks like my balance was not fetched correctly, and then subsequently defaulted to 0.

---------------------------------------------------------------------------
NotEnoughCashError                        Traceback (most recent call last)
<ipython-input-10-09c6aea41420> in <module>()
    335                                     base_currency=base,
    336                                     live_graph=False,
--> 337                                     simulate_orders=sim_orders
    338                                 )
    339 

/Users/cc/catalyst/catalyst/utils/run_algo.pyc in run_algorithm(initialize, capital_base, start, end, handle_data, before_trading_start, analyze, data_frequency, data, bundle, bundle_timestamp, default_extension, extensions, strict_extensions, environ, live, exchange_name, base_currency, algo_namespace, live_graph, analyze_live, simulate_orders, auth_aliases, stats_output, output)
    547         simulate_orders=simulate_orders,
    548         auth_aliases=auth_aliases,
--> 549         stats_output=stats_output
    550     )

/Users/cc/catalyst/catalyst/utils/run_algo.pyc in _run(handle_data, initialize, before_trading_start, analyze, algofile, algotext, defines, data_frequency, capital_base, data, bundle, bundle_timestamp, start, end, output, print_algo, local_namespace, environ, live, exchange, algo_namespace, base_currency, live_graph, analyze_live, simulate_orders, auth_aliases, stats_output)
    329     ).run(
    330         data,
--> 331         overwrite_sim_params=False,
    332     )
    333 

/Users/cc/catalyst/catalyst/exchange/exchange_algorithm.pyc in run(self, data, overwrite_sim_params)
    309         data.attempts = self.attempts
    310         return super(ExchangeTradingAlgorithmBase, self).run(
--> 311             data, overwrite_sim_params
    312         )
    313 

/Users/cc/catalyst/catalyst/algorithm.pyc in run(self, data, overwrite_sim_params)
    722         try:
    723             perfs = []
--> 724             for perf in self.get_generator():
    725                 perfs.append(perf)
    726 

/Users/cc/catalyst/catalyst/gens/tradesimulation.pyc in transform(self)
    222             for dt, action in self.clock:
    223                 if action == BAR:
--> 224                     for capital_change_packet in every_bar(dt):
    225                         yield capital_change_packet
    226                 elif action == SESSION_START:

/Users/cc/catalyst/catalyst/gens/tradesimulation.pyc in every_bar(dt_to_use, current_data, handle_data)
    135                     perf_tracker.process_commission(commission)
    136 
--> 137             handle_data(algo, current_data, dt_to_use)
    138 
    139             # grab any new orders from the blotter, then clear the list.

/Users/cc/catalyst/catalyst/utils/events.pyc in handle_data(self, context, data, dt)
    214                     context,
    215                     data,
--> 216                     dt,
    217                 )
    218 

/Users/cc/catalyst/catalyst/utils/events.pyc in handle_data(self, context, data, dt)
    233         """
    234         if self.rule.should_trigger(dt):
--> 235             self.callback(context, data)
    236 
    237 

/Users/cc/catalyst/catalyst/exchange/exchange_algorithm.pyc in handle_data(self, data)
    815                 sleeptime=self.attempts['retry_sleeptime'],
    816                 retry_exceptions=(ExchangeRequestError,),
--> 817                 cleanup=lambda: log.warn('Ordering again.')
    818             )
    819             self.portfolio_needs_update = False

/anaconda/envs/catalyst/lib/python2.7/site-packages/redo/__init__.pyc in retry(action, attempts, sleeptime, max_sleeptime, sleepscale, jitter, retry_exceptions, cleanup, args, kwargs)
    160             logfn = log.info if n != 1 else log.debug
    161             logfn(log_attempt_format, n)
--> 162             return action(*args, **kwargs)
    163         except retry_exceptions:
    164             log.debug("retry: Caught exception: ", exc_info=True)

/Users/cc/catalyst/catalyst/exchange/exchange_algorithm.pyc in synchronize_portfolio(self)
    653                 open_orders=orders,
    654                 check_balances=check_balances,
--> 655                 cash=required_cash,
    656             )
    657             total_cash += cash

/Users/cc/catalyst/catalyst/exchange/exchange.pyc in sync_positions(self, positions, open_orders, cash, check_balances)
    711                         exchange=self.name,
    712                         free=free_cash,
--> 713                         cash=cash,
    714                     )
    715 

NotEnoughCashError: Total usdt amount on bittrex is lower than the cash reserved for this algo: 0.0 < 86.31077346. While trades can be made on the exchange accounts outside of the algo, exchange must have enough free usdt to cover the algo cash.

[frisitano]

A potential solution to this would be to ensure that the required balances have been fetched correctly before performing the balance validation. The balances appear to be fetched in the sync_positions() function in exchange.py. Below I have added some code to indicate how this could potentially be achieved. The code doesn't conform to Catalyst's style so I don't expect it to be included, it's just meant to serve as a proof of concept.

def sync_positions(self, positions, open_orders=None, cash=None,
                   check_balances=False):
    """
    Update the portfolio cash and position balances based on the
    latest ticker prices.

    Parameters
    ----------
    positions:
        The positions to synchronize.

    check_balances:
        Check balances amounts against the exchange.

    """
    if check_balances:
        required_balances = [self.base_currency] + \
                            [position.asset.base_currency for position in positions]
        retry_limit = 5
        for attempt in range(retry_limit):
            balances_aquired_succesfully = True
            balances = self.api.get_balances()
            for currency in required_balances:
                if currency not in balances:
                    balances_aquired_succesfully = False
            if balances_aquired_succesfully:
                break

        if not balances_aquired_succesfully:
            raise Exception('Balances not aquired succesfully!')

[AvishaiW]

Hi @zackgow, it seems that you had some temporary connectivity issues which led to difficulties getting the balance data. It seems like in your case @frisitano's code, might be helpful. please inform us if the error continues to appear, and if so, we will try to make a change similar to @frisitano's offer, that would be suitable.

@frisitano Thank you for your contribution!

[AvishaiW]

closing. please feel free to reopen if errors still occur.

[ghost]

Hi, @AvishaiW. I'm getting exactly the same error as @zackgow when trying to trade BTC/USDT on bittrex, although my algos have been trading on Bitfinex for days. I tried @frisitano's solution with no success.

[AvishaiW]

@arthrfrnco try updating catalyst to it's newest version (0.5.9) and check if it still occurs.