error with bt.algos.WeighERC()

[summorwenluo]

There is Error related to bt.algos.WeighERC(). Just write the following codes.

import bt as bt import pandas as pd import numpy as np import matplotlib as mpl

fetch some data_bt

data_bt = bt.get('aapl,msft,c,gs,ge', start='2010-01-01') data_bt.head()

create the strategy - s1

s1 = bt.Strategy('s1', [bt.algos.RunMonthly(), bt.algos.SelectAll(), bt.algos.WeighERC(), bt.algos.Rebalance()])

create a backtest and run it

test = bt.Backtest(s1, data_bt) res = bt.run(test)

ZeroDivisionError: float division by zero

ValueError: No solution found after 100 iterations.

May I get any help? Thank you.

[boyac]

haven't been using ERC() yet, but there is also an example from the package. Please try this, it works for me, use bt.algos.RunAfterDays(n) instead of RunMonthly() RunAfterDays() is useful for algos that rely on trailing averages where you don't want to start trading until some amount of data has been built up

import bt as bt
import pandas as pd
import numpy as np
import matplotlib as mpl

# fetch some data_bt
data_bt = bt.get('aapl,msft,c,gs,ge', start='2010-01-01')
data_bt.head()

# create the strategy - s1
s1 = bt.Strategy('s1', [bt.algos.RunAfterDays(123),
                        bt.algos.SelectAll(),
                        bt.algos.WeighERC(),
                        bt.algos.Rebalance()])

# create a backtest and run it
test = bt.Backtest(s1, data_bt)
res = bt.run(test)
res.plot()

[mirca]

@summorwenluo you might want to increase the number of iterations, like:

s1 = bt.Strategy('s1', [bt.algos.RunMonthly(),
bt.algos.SelectAll(),
bt.algos.WeighERC(maximum_iterations = 1000),
bt.algos.Rebalance()])

The classical ERC computation is a convex problem solved by iterative algorithms that in principle should converge to the global optimal solution with enough number of iterations.

[mirca]

@summorwenluo I did a pretty simple example here I can confirm that increasing the number of iterations doesn't help, so there may be a legit bug in the ERC portfolio implementation.

[FinQuest]

I am also able to successfully run the example specified above by @boyac. The algorithm should be fine with the default iterations and will still converge with maximum_iterations == 25. (You may want to keep bt.algos.RunMonthly() after bt.algos.RunAfterDays(123) to lower the rebalance frequency).

Are you able to run the same example successfully using bt.algos.WeighMeanVar()?

[JordanPlatts]

To debug you can put it in pycharm, go to the time when it fails pause the debugger and see if there is anything wrong with your data at that moment. It's unlikely the fault is in the scipy optimize package that is widely used.

[mirca]

@FinQuest no luck with bt.algos.WeighMeanVar() either.

@JordanPlatts the 'ccd' implementation for the ERC portfolio is done in ffn, so the failure seems legit.

[mirca]

@JordanPlatts check out this minimal example:

import bt
data = bt.get('spy,agg', start='2010-01-01')
s = bt.Strategy('s1', [bt.algos.RunMonthly(),
                       bt.algos.SelectAll(),
                       bt.algos.WeighERC(covar_method='standard', maximum_iterations = 10000),
                       bt.algos.Rebalance()])
test = bt.Backtest(s, data)
res = bt.run(test)

which gives me the following error:

/Users/mirca/miniconda3/lib/python3.7/site-packages/numpy/lib/function_base.py:390: RuntimeWarning: Mean of empty slice.
  avg = a.mean(axis)
/Users/mirca/miniconda3/lib/python3.7/site-packages/numpy/core/_methods.py:154: RuntimeWarning: invalid value encountered in true_divide
  ret, rcount, out=ret, casting='unsafe', subok=False)
/Users/mirca/miniconda3/lib/python3.7/site-packages/pandas/core/frame.py:7639: RuntimeWarning: Degrees of freedom <= 0 for slice
  baseCov = np.cov(mat.T)
/Users/mirca/miniconda3/lib/python3.7/site-packages/numpy/lib/function_base.py:2455: RuntimeWarning: divide by zero encountered in true_divide
  c *= np.true_divide(1, fact)
/Users/mirca/miniconda3/lib/python3.7/site-packages/numpy/lib/function_base.py:2455: RuntimeWarning: invalid value encountered in multiply
  c *= np.true_divide(1, fact)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-46f9985afc64> in <module>
      1 test = bt.Backtest(s, data)
----> 2 res = bt.run(test)

~/dev/bt/bt/backtest.py in run(*backtests)
     26     # run each backtest
     27     for bkt in backtests:
---> 28         bkt.run()
     29 
     30     return Result(*backtests)

~/dev/bt/bt/backtest.py in run(self)
    199 
    200             if not self.strategy.bankrupt:
--> 201                 self.strategy.run()
    202                 # need update after to save weights, values and such
    203                 self.strategy.update(dt)

~/dev/bt/bt/core.cpython-37m-darwin.so in bt.core.Strategy.run()

~/dev/bt/bt/core.cpython-37m-darwin.so in bt.core.AlgoStack.__call__()

~/dev/bt/bt/algos.py in __call__(self, target)
    971             risk_parity_method=self.risk_parity_method,
    972             maximum_iterations=self.maximum_iterations,
--> 973             tolerance=self.tolerance)
    974 
    975         target.temp['weights'] = tw.dropna()

~/dev/ffn/ffn/core.py in calc_erc_weights(returns, initial_weights, risk_weights, covar_method, risk_parity_method, maximum_iterations, tolerance)
   1677             risk_weights,
   1678             maximum_iterations,
-> 1679             tolerance
   1680         )
   1681     elif risk_parity_method == 'slsqp':

~/dev/ffn/ffn/core.py in _erc_weights_ccd(x0, cov, b, maximum_iterations, tolerance)
   1618     # no solution found
   1619     raise ValueError('No solution found after {0} iterations.'.format(
-> 1620         maximum_iterations))
   1621 
   1622 

ValueError: No solution found after 10000 iterations.

[JordanPlatts]

Honesty I forget what it will do when you don't supply a look back period or a lag. Also if you don't use a lag of at least one day your going to rebalance at the start of the day using end of day returns.

My best advice is to try and copy the example but also just use pycharm to debug it.

https://github.com/pmorissette/bt/blob/master/examples/ERC.ipynb

[FinQuest]

Some of Weigh() Algos require a minimal period of time series price data to calculate the Covariance or Volatility. Depending on the Run() Algo used it may be necessary to call RunAfterDays() or RunAfterDate() first. I guess the error handling could be improved to make more obvious.