Open Viech opened 4 years ago
Apparently I cannot attach this to a comment either even if I put it in an archive. So here are links:
http://dl.viech.name/picos_issue_201.py http://dl.viech.name/picos_issue_201.pickle
% md5sum picos_issue_201.*
e915abb637117e44a14bbc221f4a8af1 picos_issue_201.pickle
e4d614a1446e84b0ca26778dc951f320 picos_issue_201.py
Please note the security warning in https://docs.python.org/3/library/pickle.html.
If you do not want to load pickled data, feel free to instead checkout the dualize2
branch of https://gitlab.com/picos-api/picos and run ./test.py -c power_trace -o dualize=True -s smcp
.
On PIOCS end, we track this in https://gitlab.com/picos-api/picos/issues/201.
This looks like a problem where the default tolerances are too strict for the default KKT solver.
The NameError
arises because kktsolver='ldl'
is not a valid choice ('chol'
(default) and 'qr'
are currently the only valid choices). The following commit "fixes" this issue by throwing a ValueError exception:
https://github.com/cvxopt/smcp/commit/86fb5f07d1f67a71efec6dc691caf73a3c852442
I tried with kktsolver="qr"
but there is a different issue:
Terminated (small step size detected).
Primal objective: -4.66745235e+00
Dual objective: -4.66745235e+00
Gap: 7.10227696e-11
Relative gap: 1.52166030e-11
Primal infeasibility: 2.32939129e+03
Dual infeasibility: 6.86684721e-01
Iterations: 76
CPU time: 6.76
CPU time per iteration: 0.09
Real time: 6.77
Real time per iteration: 0.09
I get the following output if I change 'ldl'
to 'qr'
:
Optimal solution found.
Primal objective: -4.66748488e+00
Dual objective: -4.66748488e+00
Gap: 3.56810782e-08
Relative gap: 7.64460498e-09
Primal infeasibility: 7.57010386e-09
Dual infeasibility: 3.36551174e-09
Iterations: 21
CPU time: 6.27
CPU time per iteration: 0.30
Real time: 3.17
Real time per iteration: 0.15
Which version of SMCP and Chompack are you using?
Arch Linux 5.4.15
Python 3.8.1
SMCP 0.4.6
Chompack 2.3.2
Same issue when updating to Chompack 2.3.3.
My guess is that it is a numerically challenging problem for SMCP with the default tolerances. SMCP is not as mature/robust as general purpose conic solvers such as CVXOPT's conelp()
and MOSEK. The difference may be explained by different BLAS libraries (leading to different round off errors). What happens if you relax the feasibility tolerance?
smcp.solvers.options['feastol'] = 1e-7 # default is 1e-8
My BLAS is 3.9.0. With 1e-7
, it works with "qr"
but still not with "chol"
. With 1e-6
it works for both factorizations.
So I guess this is nothing you can easily fix and I should update our testbench to skip some known failures by default. I leave it to you to close or leave the issue open.
Just for the sake of completeness, I noticed that failures like this can happen or not based on the order of variables passed to SMCP. So in the long run, maybe some clever reordering of the constraint matrix can improve stability.
The following script:
Produces the output:
I will attach the script and the pickled problem data in a comment, as I do not seem to be able to attach when editing the issue that I accidentially submitted with an empty description.