Description

SCS solver gives non-optimal result when running in CVXPY for semivariance portfolio optimization task library pyportfolioopt. I not sure, it is possible that there is the bug in CVXPY somethere.

How to reproduce

file test_efficient_semivariance.py from https://pyportfolioopt.readthedocs.io/en/latest/ package function test_efficient_risk changed for SCS solver to run with

if name == "main":

from tests.utilities_for_tests import setup_efficient_semivariance, get_data

from utilities_for_tests import setup_efficient_semivariance, get_data    
# es = setup_efficient_semivariance()
es = setup_efficient_semivariance(solver='SCS', verbose=True)
w = es.efficient_risk(0.2)

assert isinstance(w, dict)
assert set(w.keys()) == set(es.tickers)
# np.testing.assert_almost_equal(es.weights.sum(), 1)
np.testing.assert_almost_equal(es.weights.sum(), 1, decimal=5)
# assert all([i >= -1e-5 for i in w.values()])
assert all([i >= -1e-4 for i in w.values()])

np.testing.assert_allclose(
    es.portfolio_performance(),
    (0.4567521774612227, 0.2, 2.183678863194344),
    rtol=1e-4,
    atol=1e-4,
)

Output

SCS v2.1.2 - Splitting Conic Solver (c) Brendan O'Donoghue, Stanford University, 2012

Lin-sys: sparse-direct, nnz in A = 24954 eps = 1.00e-04, alpha = 1.50, max_iters = 5000, normalize = 1, scale = 1.00 acceleration_lookback = 0, rho_x = 1.00e-03 Variables n = 2705, constraints m = 5412 Cones: primal zero / dual free vars: 896 linear vars: 1831 soc vars: 2685, soc blks: 895 WARN: aa_init returned NULL, no acceleration applied. Setup time: 1.53e-01s

Iter | pri res | dua res | rel gap | pri obj | dua obj | kap/tau | time (s)

 0| 1.29e+21  1.91e+20  9.75e-01 -4.21e+22 -5.39e+20  1.35e+23  1.23e-01

100| 1.01e-02 9.86e-04 1.03e-01 -9.33e-01 -6.66e-01 2.16e-14 3.13e-01 200| 5.08e-03 4.67e-04 7.16e-02 -7.72e-01 -6.02e-01 2.22e-14 4.86e-01 300| 5.10e-03 1.48e-04 8.54e-02 -7.66e-01 -5.67e-01 2.28e-14 6.81e-01 400| 5.13e-03 5.78e-05 1.01e-01 -7.60e-01 -5.30e-01 2.35e-14 8.51e-01 500| 5.17e-03 2.72e-05 1.17e-01 -7.54e-01 -4.90e-01 2.43e-14 1.02e+00 600| 7.80e-06 1.55e-04 1.32e-06 -4.65e-01 -4.65e-01 1.06e-15 1.21e+00 620| 5.47e-05 6.93e-05 1.85e-06 -4.65e-01 -4.65e-01 2.49e-14 1.25e+00

Status: Solved Timing: Solve time: 1.25e+00s Lin-sys: nnz in L factor: 33451, avg solve time: 8.15e-04s Cones: avg projection time: 1.50e-04s Acceleration: avg step time: 7.27e-08s

Error metrics: dist(s, K) = 1.0445e-14, dist(y, K*) = 8.6736e-19, s'y/|s||y| = -6.2081e-17 primal res: |Ax + s - b|_2 / (1 + |b|_2) = 5.4692e-05 dual res: |A'y + c|_2 / (1 + |c|_2) = 6.9337e-05 rel gap: |c'x + b'y| / (1 + |c'x| + |b'y|) = 1.8452e-06

c'x = -0.4650, -b'y = -0.4650

bodono commented 3 years ago

Looks like you want higher accuracy here? If so you can reduce the eps quantity through cvxpy which will yield better solutions (but take more time to solve).

AndreiKeino commented 3 years ago

Thanks, but I am afraid that it's not an accuracy issue. The SCS solver ffound solution which is not exact, the solution is a way too distant and has bigger Sortino ratio - 2.5 times bigger - comparing with one I can get with ECOS solver for that task.

cvxgrp / scs

SCS solver presumably failed to find optimum of the problem with CVXPY #171