Running Backtest

[emretezel]

Hi Rob, everyone,

After upgrading to latest head in master and updating pandas etc. I am trying to run a backtest with estimations. I believe I have updated arctic data for the latest version of the packages. Getting the following error, any pointers greatly appreciated.

`from systems.provided.futures_chapter15.estimatedsystem import futures_system from sysdata.sim.db_futures_sim_data import dbFuturesSimData from sysdata.config.configdata import Config

config = Config('private.strategy.config.yaml') data = dbFuturesSimData() system = futures_system(data=data, config=config) capital = system.accounts.get_actual_capital()`

Including my config file (changed yaml to txt so that I can add to GitHub. config.txt

Getting the following error

`Calculating pandl for portfolio Calculating instrument div. multiplier Calculating instrument correlations /Users/batman/opt/anaconda3/envs/pysystemtrade/lib/python3.8/site-packages/numpy/lib/nanfunctions.py:1113: RuntimeWarning: All-NaN slice encountered r, k = function_base._ureduce(a, func=_nanmedian, axis=axis, out=out, Calculating raw forecast weights for BOBL FIXME / WARNING: get_default_config_key_value SHOULD NOT BE USED! FIXME / WARNING: get_default_config_key_value SHOULD NOT BE USED! FIXME / WARNING: get_default_config_key_value SHOULD NOT BE USED! FIXME / WARNING: get_default_config_key_value SHOULD NOT BE USED! Calculating pandl for instrument rules for BOBL Calculating pandl for instrument rules for COPPER Calculating pandl for instrument rules for CRUDE_W Calculating pandl for instrument rules for EDOLLAR Calculating pandl for instrument rules for EUR Calculating pandl for instrument rules for EUROSTX Calculating pandl for instrument rules for GAS_US Calculating pandl for instrument rules for GBP Calculating pandl for instrument rules for GOLD Calculating pandl for instrument rules for JPY Calculating pandl for instrument rules for LEANHOG Calculating pandl for instrument rules for LIVECOW Calculating pandl for instrument rules for NIKKEI Calculating pandl for instrument rules for SILVER Calculating pandl for instrument rules for SOYBEAN Calculating pandl for instrument rules for SP500 Calculating pandl for instrument rules for US10 Calculating pandl for instrument rules for WHEAT Using cost multiplier on optimisation of 0.00 * USE METHOD IN SYSQUANT INSTEAD [===----------------------------------------------------------------------------] 4.5% OptimisinggNatural top level grouping used /Users/batman/opt/pysystemtrade/syscore/handcrafting.py:438: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy instrument_returns[instrument_returns == 0.0] = np.nan /Users/batman/opt/anaconda3/envs/pysystemtrade/lib/python3.8/site-packages/pandas/core/frame.py:2986: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy self._where(-key, value, inplace=True) /Users/batman/opt/pysystemtrade/syscore/correlations.py:367: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray m = np.array(m)

ValueError Traceback (most recent call last)

in ----> 1 capital = system.accounts.get_actual_capital() ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/account.py in get_actual_capital(self, delayfill, roundpositions) 1085 """ 1086 -> 1087 capmult = self.capital_multiplier() 1088 notional = self.get_notional_capital() 1089 ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/account.py in capital_multiplier(self, delayfill, roundpositions) 1064 capmult_func = resolve_function(capmult_params.pop("func")) 1065 -> 1066 capmult = capmult_func(system, **capmult_params) 1067 1068 capmult = capmult.reindex(self.portfolio().index).ffill() ~/opt/pysystemtrade/syscore/capital.py in full_compounding(system, **ignored_args) 17 18 def full_compounding(system, **ignored_args): ---> 19 pandl = system.accounts.portfolio().percent() 20 multiplier = 1.0 + (pandl / 100.0) 21 multiplier = multiplier.cumprod().ffill() ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/account.py in portfolio(self, delayfill, roundpositions) 1282 capital = self.get_notional_capital() 1283 instruments = self.get_instrument_list() -> 1284 port_pandl = [ 1285 self.pandl_for_instrument( 1286 instrument_code, delayfill=delayfill, roundpositions=roundpositions ~/opt/pysystemtrade/systems/account.py in (.0) 1283 instruments = self.get_instrument_list() 1284 port_pandl = [ -> 1285 self.pandl_for_instrument( 1286 instrument_code, delayfill=delayfill, roundpositions=roundpositions 1287 ) ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/account.py in pandl_for_instrument(self, instrument_code, delayfill, roundpositions) 633 634 price = self.get_daily_price(instrument_code) --> 635 positions = self.get_buffered_position( 636 instrument_code, roundpositions=roundpositions 637 ) ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/accounts_inputs.py in get_buffered_position(self, instrument_code, roundpositions) 604 605 self.log.msg("Calculating buffered positions") --> 606 optimal_position = self.get_notional_position(instrument_code) 607 pos_buffers = self.get_buffers_for_position(instrument_code) 608 trade_to_edge = self.parent.config.buffer_trade_to_edge ~/opt/pysystemtrade/systems/system_cache.py in wrapper(*args, **kwargs) 706 def null_decorator(func): 707 def wrapper(*args, **kwargs): --> 708 return func(*args, **kwargs) 709 710 return wrapper ~/opt/pysystemtrade/systems/accounts_inputs.py in get_notional_position(self, instrument_code) 265 >>> 266 """ --> 267 return self.parent.portfolio.get_notional_position(instrument_code) 268 269 @input ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/portfolio.py in get_notional_position(self, instrument_code) 685 ) 686 --> 687 idm = self.get_instrument_diversification_multiplier() 688 instr_weights = self.get_instrument_weights() 689 subsys_position = self.get_subsystem_position(instrument_code) ~/opt/pysystemtrade/systems/system_cache.py in wrapper(*args, **kwargs) 706 def null_decorator(func): 707 def wrapper(*args, **kwargs): --> 708 return func(*args, **kwargs) 709 710 return wrapper ~/opt/pysystemtrade/systems/portfolio.py in get_instrument_diversification_multiplier(self) 647 648 if self.use_estimated_instrument_div_mult(): --> 649 return self.get_estimated_instrument_diversification_multiplier() 650 else: 651 return self.get_fixed_instrument_diversification_multiplier() ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/portfolio.py in get_estimated_instrument_diversification_multiplier(self) 591 idm_func = resolve_function(div_mult_params.pop("func")) 592 --> 593 correlation_list_object = self.get_instrument_correlation_matrix() 594 weight_df = self.get_instrument_weights() 595 ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/portfolio.py in get_instrument_correlation_matrix(self) 544 545 if hasattr(system, "accounts"): --> 546 pandl = self.pandl_across_subsystems().to_frame() 547 else: 548 error_msg = "You need an accounts stage in the system to estimate instrument correlations" ~/opt/pysystemtrade/systems/system_cache.py in wrapper(*args, **kwargs) 706 def null_decorator(func): 707 def wrapper(*args, **kwargs): --> 708 return func(*args, **kwargs) 709 710 return wrapper ~/opt/pysystemtrade/systems/portfolio.py in pandl_across_subsystems(self) 301 """ 302 --> 303 return self.parent.accounts.pandl_across_subsystems() 304 305 ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/account.py in pandl_across_subsystems(self, delayfill, roundpositions) 586 587 instruments = self.get_instrument_list() --> 588 pandl_across_subsys = [ 589 self.pandl_for_subsystem( 590 instrument_code, delayfill=delayfill, roundpositions=roundpositions ~/opt/pysystemtrade/systems/account.py in (.0) 587 instruments = self.get_instrument_list() 588 pandl_across_subsys = [ --> 589 self.pandl_for_subsystem( 590 instrument_code, delayfill=delayfill, roundpositions=roundpositions 591 ) ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/account.py in pandl_for_subsystem(self, instrument_code, delayfill, roundpositions) 524 525 price = self.get_daily_price(instrument_code) --> 526 positions = self.get_aligned_subsystem_position(instrument_code) 527 528 fx = self.get_fx_rate(instrument_code) ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/accounts_inputs.py in get_aligned_subsystem_position(self, instrument_code) 367 """ 368 price = self.get_daily_price(instrument_code) --> 369 sspos = self.get_subsystem_position(instrument_code) 370 371 sspos = sspos.reindex(price.index).ffill() ~/opt/pysystemtrade/systems/system_cache.py in wrapper(*args, **kwargs) 706 def null_decorator(func): 707 def wrapper(*args, **kwargs): --> 708 return func(*args, **kwargs) 709 710 return wrapper ~/opt/pysystemtrade/systems/accounts_inputs.py in get_subsystem_position(self, instrument_code) 133 134 """ --> 135 return self.parent.positionSize.get_subsystem_position(instrument_code) 136 137 @input ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/positionsizing.py in get_subsystem_position(self, instrument_code) 87 avg_abs_forecast = self.avg_abs_forecast() 88 vol_scalar = self.get_volatility_scalar(instrument_code) ---> 89 forecast = self.get_combined_forecast(instrument_code) 90 91 vol_scalar = vol_scalar.reindex(forecast.index).ffill() ~/opt/pysystemtrade/systems/system_cache.py in wrapper(*args, **kwargs) 706 def null_decorator(func): 707 def wrapper(*args, **kwargs): --> 708 return func(*args, **kwargs) 709 710 return wrapper ~/opt/pysystemtrade/systems/positionsizing.py in get_combined_forecast(self, instrument_code) 524 """ 525 --> 526 return self.comb_forecast_stage.get_combined_forecast(instrument_code) 527 528 @property ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/forecast_combine.py in get_combined_forecast(self, instrument_code) 68 instrument_code=instrument_code, 69 ) ---> 70 raw_multiplied_combined_forecast = self.get_raw_combined_forecast_before_mapping( 71 instrument_code 72 ) ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/forecast_combine.py in get_raw_combined_forecast_before_mapping(self, instrument_code) 89 90 # sum ---> 91 raw_combined_forecast = self.get_combined_forecast_without_multiplier(instrument_code) 92 93 forecast_div_multiplier = self.get_forecast_diversification_multiplier( ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/forecast_combine.py in get_combined_forecast_without_multiplier(self, instrument_code) 109 # might be that some rules were omitted in the weight calculation 110 --> 111 weighted_forecasts = self.get_weighted_forecasts_without_multiplier(instrument_code) 112 113 # sum ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/forecast_combine.py in get_weighted_forecasts_without_multiplier(self, instrument_code) 124 instrument_code, rule_variation_list) 125 --> 126 smoothed_monthly_forecast_weights = self.get_forecast_weights(instrument_code) 127 smoothed_forecast_weights = smoothed_monthly_forecast_weights.reindex(forecasts.index).ffill() 128 ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/forecast_combine.py in get_forecast_weights(self, instrument_code) 136 def get_forecast_weights(self, instrument_code: str) -> pd.DataFrame: 137 # These will be in monthly frequency --> 138 monthly_forecast_weights_fixed_to_forecasts_unsmoothed = self.get_unsmoothed_forecast_weights(instrument_code) 139 140 # smooth out weights ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/forecast_combine.py in get_unsmoothed_forecast_weights(self, instrument_code) 194 # from fixed weights 195 # These are monthly to save space --> 196 monthly_forecast_weights = self.get_raw_monthly_forecast_weights(instrument_code) 197 198 # fix to forecast time series ~/opt/pysystemtrade/systems/system_cache.py in wrapper(*args, **kwargs) 706 def null_decorator(func): 707 def wrapper(*args, **kwargs): --> 708 return func(*args, **kwargs) 709 710 return wrapper ~/opt/pysystemtrade/systems/forecast_combine.py in get_raw_monthly_forecast_weights(self, instrument_code) 459 # get raw weights 460 if self._use_estimated_weights(): --> 461 forecast_weights = self.get_monthly_raw_forecast_weights_estimated( 462 instrument_code) 463 else: ~/opt/pysystemtrade/systems/forecast_combine.py in get_monthly_raw_forecast_weights_estimated(self, instrument_code) 508 """ 509 #FIXME REFACTOR --> 510 return self.calculation_of_raw_estimated_monthly_forecast_weights( 511 instrument_code 512 ).weights ~/opt/pysystemtrade/systems/system_cache.py in wrapper(self, *args, **kwargs) 729 this_stage = self 730 --> 731 ans = system.cache.calc_or_cache( 732 func, 733 this_stage, ~/opt/pysystemtrade/systems/system_cache.py in calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, *args, **kwargs) 587 # call the function. Note in the original function 'this_stage' was 588 # 'self' --> 589 value = func(this_stage, *args, **kwargs) 590 self.set_item_in_cache( 591 value, ~/opt/pysystemtrade/systems/forecast_combine.py in calculation_of_raw_estimated_monthly_forecast_weights(self, instrument_code) 561 **weighting_params) 562 --> 563 weight_func.optimise() 564 565 return weight_func ~/opt/pysystemtrade/syscore/optimisation.py in optimise(self) 292 # Do the optimisation for one period, using a particular optimiser 293 # instance --> 294 results_this_period = optSinglePeriod( 295 self, data, fit_period, optimiser, cleaning 296 ) ~/opt/pysystemtrade/syscore/optimisation.py in __init__(self, parent, data, fit_period, optimiser, cleaning) 769 subset_fitting_data = data[fit_period.fit_start: fit_period.fit_end] 770 --> 771 (weights, diag) = optimiser.call( 772 subset_fitting_data, cleaning, must_haves) 773 ~/opt/pysystemtrade/syscore/optimisation.py in call(self, optimise_data, cleaning, must_haves) 732 733 params = self.params --> 734 return self.opt_func( 735 optimise_data, 736 self.moments_estimator, ~/opt/pysystemtrade/syscore/optimisation.py in handcraft(period_subset_data, moments_estimator, cleaning, must_haves, equalise_SR, equalise_vols, **ignored_args) 964 965 if equalise_vols: --> 966 unclean_weights = portfolio.volatility_weights_with_missing_data() 967 else: 968 unclean_weights = portfolio.cash_weights_with_missing_data() ~/opt/pysystemtrade/syscore/handcrafting.py in volatility_weights_with_missing_data(self) 1186 """ 1187 -> 1188 vol_weights_valid_only = self.volatility_weights 1189 vol_weights = self._weights_with_missing_data(vol_weights_valid_only) 1190 ~/opt/pysystemtrade/syscore/handcrafting.py in volatility_weights(self) 1211 return self._volatility_weights 1212 else: -> 1213 weights_vol = self._calculate_volatility_weights() 1214 self._volatility_weights = weights_vol 1215 ~/opt/pysystemtrade/syscore/handcrafting.py in _calculate_volatility_weights(self) 950 vol_weights = self._calculate_weights_standalone_portfolio() 951 else: --> 952 vol_weights = self._calculate_weights_aggregated_portfolio() 953 954 return vol_weights ~/opt/pysystemtrade/syscore/handcrafting.py in _calculate_weights_aggregated_portfolio(self) 852 """ 853 --> 854 sub_portfolio_returns = self._calculate_sub_portfolio_returns() 855 856 # create another Portfolio object made up of the sub portfolios ~/opt/pysystemtrade/syscore/handcrafting.py in _calculate_sub_portfolio_returns(self) 837 assert self.sub_portfolios is not NO_SUB_PORTFOLIOS 838 --> 839 sub_portfolio_returns = [ 840 sub_portfolio.portfolio_returns for sub_portfolio in self.sub_portfolios] 841 sub_portfolio_returns = pd.concat(sub_portfolio_returns, axis=1) ~/opt/pysystemtrade/syscore/handcrafting.py in (.0) 838 839 sub_portfolio_returns = [ --> 840 sub_portfolio.portfolio_returns for sub_portfolio in self.sub_portfolios] 841 sub_portfolio_returns = pd.concat(sub_portfolio_returns, axis=1) 842 ~/opt/pysystemtrade/syscore/handcrafting.py in portfolio_returns(self) 1241 return self._portfolio_returns 1242 else: -> 1243 portfolio_returns = self._calculate_portfolio_returns() 1244 self._portfolio_returns = portfolio_returns 1245 ~/opt/pysystemtrade/syscore/handcrafting.py in _calculate_portfolio_returns(self) 795 """ 796 --> 797 cash_weights = self.cash_weights 798 instrument_returns = self.instrument_returns 799 ~/opt/pysystemtrade/syscore/handcrafting.py in cash_weights(self) 1221 return self._cash_weights 1222 else: -> 1223 weights_cash = self._calculate_cash_weights() 1224 self._cash_weights = weights_cash 1225 ~/opt/pysystemtrade/syscore/handcrafting.py in _calculate_cash_weights(self) 1144 if target_std is NO_RISK_TARGET: 1145 # no risk target, can use natural weights -> 1146 return self._calculate_cash_weights_no_risk_target() 1147 elif self.top_level_weights is not NO_TOP_LEVEL_WEIGHTS: 1148 # top level weights passed, use natural weights ~/opt/pysystemtrade/syscore/handcrafting.py in _calculate_cash_weights_no_risk_target(self) 960 :return: list of cash weights 961 """ --> 962 vol_weights = self.volatility_weights 963 instrument_std = self.vol_vector 964 ~/opt/pysystemtrade/syscore/handcrafting.py in volatility_weights(self) 1211 return self._volatility_weights 1212 else: -> 1213 weights_vol = self._calculate_volatility_weights() 1214 self._volatility_weights = weights_vol 1215 ~/opt/pysystemtrade/syscore/handcrafting.py in _calculate_volatility_weights(self) 948 949 if self.sub_portfolios is NO_SUB_PORTFOLIOS: --> 950 vol_weights = self._calculate_weights_standalone_portfolio() 951 else: 952 vol_weights = self._calculate_weights_aggregated_portfolio() ~/opt/pysystemtrade/syscore/handcrafting.py in _calculate_weights_standalone_portfolio(self) 768 assert self.sub_portfolios is NO_SUB_PORTFOLIOS 769 --> 770 raw_weights = get_weights_using_uncertainty_method( 771 self.corr_matrix.values, len(self.instrument_returns.index)) 772 self.raw_weights = raw_weights ~/opt/pysystemtrade/syscore/handcrafting.py in get_weights_using_uncertainty_method(cmatrix, data_points) 64 raise Exception("Cluster too big") 65 ---> 66 average_weights = optimised_weights_given_correlation_uncertainty(cmatrix, data_points) 67 weights = apply_min_weight(average_weights) 68 ~/opt/pysystemtrade/syscore/handcrafting.py in optimised_weights_given_correlation_uncertainty(corr_matrix, data_points, p_step) 76 for conf3 in dist_points: 77 conf_intervals = labelledCorrelations(conf1, conf2, conf3) ---> 78 weights = optimise_for_corr_matrix_with_uncertainty(corr_matrix, conf_intervals, data_points) 79 list_of_weights.append(weights) 80 ~/opt/pysystemtrade/syscore/handcrafting.py in optimise_for_corr_matrix_with_uncertainty(corr_matrix, conf_intervals, data_points) 90 labelled_correlation_points = calculate_correlation_points_from_tuples(labelled_correlations, conf_intervals, data_points) 91 corr_matrix_at_distribution_point = three_asset_corr_matrix(labelled_correlation_points) ---> 92 weights = optimise_for_corr_matrix(corr_matrix_at_distribution_point) 93 94 return weights ~/opt/pysystemtrade/syscore/handcrafting.py in optimise_for_corr_matrix(corr_matrix) 181 std = [.1]*3 182 --> 183 return optimise_using_correlation(mean_list, corr_matrix, std) 184 185 ~/opt/pysystemtrade/syscore/handcrafting.py in optimise_using_correlation(mean_list, avg_correlation, std) 331 sigma = sigma_from_corr_and_std(stdev_list, corr_matrix) 332 --> 333 return optimise(sigma, mean_list) 334 335 ~/opt/pysystemtrade/syscore/optimisation_utils.py in optimise(sigma, mean_list) 223 224 # replaces nans with zeros --> 225 sigma = fix_sigma(sigma) 226 227 mus = np.array(mean_list, ndmin=2).transpose() ~/opt/pysystemtrade/syscore/optimisation_utils.py in fix_sigma(sigma) 210 return x 211 --> 212 sigma = [[_fixit(x) for x in sigma_row] for sigma_row in sigma] 213 214 sigma = np.array(sigma) ~/opt/pysystemtrade/syscore/optimisation_utils.py in (.0) 210 return x 211 --> 212 sigma = [[_fixit(x) for x in sigma_row] for sigma_row in sigma] 213 214 sigma = np.array(sigma) ~/opt/pysystemtrade/syscore/optimisation_utils.py in (.0) 210 return x 211 --> 212 sigma = [[_fixit(x) for x in sigma_row] for sigma_row in sigma] 213 214 sigma = np.array(sigma) ~/opt/pysystemtrade/syscore/optimisation_utils.py in _fixit(x) 205 206 def _fixit(x): --> 207 if np.isnan(x): 208 return 0.0 209 else: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() `

[robcarver17]

Since I'm currently refactoring this code it's my hope that this will be fixed when I commit what I'm doing in a couple of weeks or so.

[emretezel]

Thanks Rob, what would you recommend as the stable most recent commit?

[robcarver17]

Sorry this took so long. Had a blog post to write. Can you try the last commit, see if it works?

Ta

[robcarver17]

I'm closing this, as the code where the error exists no longer exists...

[emretezel]

Thank you Rob, it is working.

Best, Emre

On 16 Apr 2021, at 14:32, Robert Carver @.***> wrote:

Closed #323 https://github.com/robcarver17/pysystemtrade/issues/323.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/robcarver17/pysystemtrade/issues/323#event-4604879775, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEJUDINSZKASAQAKPRBCUTTJA4APANCNFSM42L2VHMQ.