Better error message for expected returns functions

[MaxMA2000]

Describe the bug I am using my stock price datasets to build up the portfolio with the mean-variance optimization, however, it cannot work. It outputs "Workspace allocation error!", and a more detailed error message is shown below. I have also created a code example with my dataset, it would be great if you can help me fix this bug soon. Thank you so much!

Expected behavior I am expecting to have a normal outcome for the weights = ef.max_sharpe()

Code sample

• dataset

import pandas as pd

df = pd.read_csv('testing.csv',index_col=0)
# Estimate the expected returns and covariance matrix from the historical data
from pypfopt.expected_returns import mean_historical_return
from pypfopt.risk_models import CovarianceShrinkage

mu = mean_historical_return(df)
S = CovarianceShrinkage(df).ledoit_wolf()

# Mean-Variance Optimization: find the portfolio that maximises the Sharpe Ratio
from pypfopt.efficient_frontier import EfficientFrontier

ef = EfficientFrontier(mu, S)
weights = ef.max_sharpe()
cleaned_weights = ef.clean_weights()

print("Portfolio Weight:" + str(dict(cleaned_weights)))

# Print out the expected performance of the portfolio
annual_sharpe_ratio = ef.portfolio_performance(verbose=True)[-1]

weight = dict(cleaned_weights)

portfolio_name, portfolio_weight = list(weight.keys()), list(weight.values())

Operating system, python version, PyPortfolioOpt version e.g MacOS 12.3.1, python 3.9, PyPortfolioOpt latest

Details of error message

ERROR in LDL_factor: Error in KKT matrix LDL factorization when computing the nonzero elements. The problem seems to be non-convex
ERROR in osqp_setup: KKT matrix factorization.
The problem seems to be non-convex.

ValueError                                Traceback (most recent call last)
Input In [56], in <module>
     12 from pypfopt.efficient_frontier import EfficientFrontier
     14 ef = EfficientFrontier(mu, S)
---> 15 weights = ef.max_sharpe()
     16 cleaned_weights = ef.clean_weights()
     18 print("Portfolio Weight:" + str(dict(cleaned_weights)))

File ~/mambaforge/lib/python3.9/site-packages/pypfopt/efficient_frontier/efficient_frontier.py:288, in EfficientFrontier.max_sharpe(self, risk_free_rate)
    281 # Transformed max_sharpe convex problem:
    282 self._constraints = [
    283     (self.expected_returns - risk_free_rate).T @ self._w == 1,
    284     cp.sum(self._w) == k,
    285     k >= 0,
    286 ] + new_constraints
--> 288 self._solve_cvxpy_opt_problem()
    289 # Inverse-transform
    290 self.weights = (self._w.value / k.value).round(16) + 0.0

File ~/mambaforge/lib/python3.9/site-packages/pypfopt/base_optimizer.py:292, in BaseConvexOptimizer._solve_cvxpy_opt_problem(self)
    287         if not constr_ids == self._initial_constraint_ids:
    288             raise exceptions.InstantiationError(
    289                 "The constraints were changed after the initial optimization. "
    290                 "Please create a new instance instead."
    291             )
--> 292     self._opt.solve(
    293         solver=self._solver, verbose=self._verbose, **self._solver_options
    294     )
    296 except (TypeError, cp.DCPError) as e:
    297     raise exceptions.OptimizationError from e

File ~/mambaforge/lib/python3.9/site-packages/cvxpy/problems/problem.py:481, in Problem.solve(self, *args, **kwargs)
    479 else:
    480     solve_func = Problem._solve
--> 481 return solve_func(self, *args, **kwargs)

File ~/mambaforge/lib/python3.9/site-packages/cvxpy/problems/problem.py:1012, in Problem._solve(self, solver, warm_start, verbose, gp, qcp, requires_grad, enforce_dpp, ignore_dpp, **kwargs)
   1008     s.LOGGER.info(
   1009             'Invoking solver %s  to obtain a solution.',
   1010             solving_chain.reductions[-1].name())
   1011 start = time.time()
-> 1012 solution = solving_chain.solve_via_data(
   1013     self, data, warm_start, verbose, kwargs)
   1014 end = time.time()
   1015 self._solve_time = end - start

File ~/mambaforge/lib/python3.9/site-packages/cvxpy/reductions/solvers/solving_chain.py:361, in SolvingChain.solve_via_data(self, problem, data, warm_start, verbose, solver_opts)
    325 def solve_via_data(self, problem, data, warm_start: bool = False, verbose: bool = False,
    326                    solver_opts={}):
    327     """Solves the problem using the data output by the an apply invocation.
    328 
    329     The semantics are:
   (...)
    359         a Solution object.
    360     """
--> 361     return self.solver.solve_via_data(data, warm_start, verbose,
    362                                       solver_opts, problem._solver_cache)

File ~/mambaforge/lib/python3.9/site-packages/cvxpy/reductions/solvers/qp_solvers/osqp_qpif.py:100, in OSQP.solve_via_data(self, data, warm_start, verbose, solver_opts, solver_cache)
     98     solver_opts['polish'] = solver_opts.get('polish', True)
     99     solver = osqp.OSQP()
--> 100     solver.setup(P, q, A, lA, uA, verbose=verbose, **solver_opts)
    102 results = solver.solve()
    104 if solver_cache is not None:

File ~/mambaforge/lib/python3.9/site-packages/osqp/interface.py:37, in OSQP.setup(self, P, q, A, l, u, **settings)
     34 self._derivative_cache = {'P': P, 'q': q, 'A': A, 'l': l, 'u': u}
     36 unpacked_data, settings = utils.prepare_data(P, q, A, l, u, **settings)
---> 37 self._model.setup(*unpacked_data, **settings)

ValueError: Workspace allocation error!

[MaxMA2000]

The problem seems to be the data source, there is a negative stock price inside one of the columns...., it should be the wrong data source.

Maybe the package can add a tiny check for the input dataset in the mu = mean_historical_return(df) in the future? That would help others with similar situations lol.

Thank you so much for the great package!

[robertmartin8]

You're right, it would be helpful if there were a little check for negative stock prices and NaNs etc.

Will leave this open until I fix it.