Add COPT support for YALMIP

[riis-hhd]

This PR adds COPT support for YALMIP, you can download the MEX binaries of COPT-MATLAB from https://github.com/leavesgrp/COPT-MATLAB/releases and follow the userguide for setup instructions.

If you want to solve larger instances via COPT, a license is required and you can obtain one from https://www.shanshu.ai/copt.

If you have any questions, feel free to contact me.

Thanks!

[johanlofberg]

Looks very solid! I will do some testing locally, and then hopefully this will make it to a new version that I plan to release within a week or so

[johanlofberg]

To make it less reliant of updates of YALMIP when you or copt updates options logics, a preferred approach is that you distribute copt-matlab with a function called something like coptoptiondefaults which sets up a default options structure, and then in sdpsettings you simply have something like

function coptops = setup_copt_options
try
    coptops = coptoptiondefaults;
catch
    coptops = [];
end

That way, YALMIP does not have to be updated when copt-matlab is updated w.r.t options

It is fine with the current approach too (unfortunately that is the most options logics are done today), just a tip.

[riis-hhd]

Thanks for your tips! I am not sure about 'coptops'? Can you explain a little bit more. I will try to research on how to do that and update in next release of COPT-MATLAB.

[johanlofberg]

coptops is just the local argument name I used in the example code

You can look at for instance the logic in setup_mosek_options in sdpsettings (not 100% clean, but close to what I mean) or setup_baron_options which is exactly what I mean. A function creating a default options structure is available if the solver (or MATLAB solver interface to be precise) is installed, and thus YALMIP just uses that.

It is rather convenient also for people not using YALMIP as it allows them to quickly understand which options are available and what their default values are

[johanlofberg]

Is there a reason for running the pruneOptions command in yalmip2copt (i.e. performance issues in your interface to parse them)?

Turned out that it caused a crash when copt is used as a local solver in the global solver bmibnb

x = sdpvar(3,1);
sol = optimize([sum(x.^3)==2, -1 <= x <= 1],-x'*x,sdpsettings('solver','bmibnb','bmibnb.lowersolver','copt','debug',1));

Not caused by your code directly, more that YALMIP assumes the pruneOptions is something which is done on a higher level (typically in bnb and bmibnb which will call the solver repeatedly) so some assumption in the code was violated

[riis-hhd]

Oops, I added the pruneOptions by following steps in CPlex, not for performance purpose. Please fix it at your convenience.

[johanlofberg]

Ok, (yes I realized when thinking about it that it only can slow down performance when placed in yalmip2copt as it has to loop through all options, which reasonably should be faster to do inside copt) hence it will be deleted

[riis-hhd]

Got it! :)

[johanlofberg]

I note you have defined QP/MIQP by declaring SOCP support only, i.e. forcing YALMIP to write all QPs as SOCPS. Is that intentional and preferred?

[riis-hhd]

COPT supports QP and MIQP. However, I do not know how to let YALMIP pass problem to copt directly. Therefore, forcing YALMIP to write to SOCPs is my workaround. I would like to know the better approach. Thanks!

Below is an example to use copt-matlab to solve QP: https://github.com/leavesgrp/COPT-MATLAB/blob/master/examples/demo_qp.m

[johanlofberg]

Ok, updated some stuff and pushed this to develop

I can trigger seg-faults with a very simple MISOCP with quadratic objective (does not happen every time, so a loop to trigger it)

x = [sdpvar(2,1);intvar(1)];
for i = 1:20;optimize([x'*x<= 5],(x-1.2)'*(x-1.2),sdpsettings('solver','copt','debug',1));end

Crashes after displaying

...
The original problem has:
    4 rows, 7 columns and 7 non-zero elements
    1 integers
    3 quadratic objective elements
    1 cones

Presolving the problem

When it runs, it computes the wrong solution (1,1,1), correct is (1.2,1.2,1)

Feel free to take further discussions via email johan.lofberg

[johanlofberg]

It appears it is not just the combination of MISOCPs and quadratic objectives, but quadratic objectives appear to easily lead to issues. If I just add the Q-matrix to the model (also when it is sparse all zeros) it often crashes

The following model from yalmiptest for instance crashes

N = 5;
A = [2 -1;1 0];
B = [1;0];
C = [0.5 0.5];
[H,S] = create_CHS(A,B,C,N);
x = [2;0];
t = sdpvar(2*N,1);
U = sdpvar(N,1);
Y = H*x+S*U;
F = (U<=1)+(U>=-1);
F = F+(Y(N)>=-1);
F = F+(Y(N)<=1);
F = F+([Y;U]<=t)+([Y;U]>=-t);
sol = optimize(F,sum(t),ops)

If I randomly add

model.Q = model.Q + speye(length(model.Q));

before the call to copt_solve, no seg-faults occur, so it appears the interface, or the solver, has issues with zero row/columns in Q

[riis-hhd]

Here are replies from COPT engineer:

1. Crashes in MISOCP model is know issue, and will be fixed in comming release (should be this week)
2. Wrong solution is due to infinity settings in copt-matlab.
3. Q matrix requires at least one non-zero element (by design) I'll fix infinity settings in copt-matlab tomorrow. Can we have more tests with new version of copt and copt-matlab later?

[johanlofberg]

2. What does that mean?
3. Aha, I actually added that as a hack already, so pushed that now

[riis-hhd]

2. In copt-matlab, it does not convert infinity to COPT_INFINITY (which is 1e30)

[johanlofberg]

OK.

Pushed some more fixes

[johanlofberg]

Another suggestion would be to not print the values of all supplied options, but only those which are different from default. that's how most (all?) other solvers behave

[riis-hhd]

That's good point. I'll change to print non-default values only.

[riis-hhd]

Hi, Johan. Can you please help to test new release of COPT and COPT-MATLAB. Crashes of MISOCP problem should be fixed by COPT v6.5.4. In addition, COPT-MATLAB handles infinity issue properly; print non-default values only; add COPT default parameters as well.

[johanlofberg]

Will test tonight

[johanlofberg]

works like a charm

Who do I acknowledge when I release this. Name or githandle or something you'd like to use?

[riis-hhd]

Thanks, Johan. COPT-MATLAB is a research project owned by Research Institute for Interdisciplinary Sciences at Shanghai University of Finance and Economics (RIIS @ Shufe). Its github site is https://github.com/leavesgrp/COPT-MATLAB. :)

[johanlofberg]

Officially released today https://yalmip.github.io/R20230622