Using Cuts produce noisy output (model.logLevel=0) -> desired behaviour?

[sschnug]

Hi,

i just implemented a prototype interface to Cbc for cvxpy using cylp (great project).

When using this to solve some naive tsp-model without using any cuts, loglevel=0 works and there is no output.

Activating LiftProjectCuts on the other hand results in the following (shortened) output:

Clp3003W Analysis indicates model infeasible or unbounded
Clp6003E Matrix has 144 large values, first at column 20, row 110 is -1.7976931e+308
Clp0004I Stopped due to errors - objective value 0
Clp0032I Errors objective 0 - 0 iterations time 0.002
Clp6003E Matrix has 144 large values, first at column 20, row 110 is -1.7976931e+308
Clp3003W Analysis indicates model infeasible or unbounded
Clp6003E Matrix has 144 large values, first at column 20, row 110 is -1.7976931e+308
Clp0004I Stopped due to errors - objective value 0
Clp0032I Errors objective 0 - 0 iterations time 0.002

Additional example with CliqueCuts:

rcl Found 0 new violated cliques with the row-clique method
rcl The largest admissible number was 0 (threshold 12)
rcl    all row cliques have been enumerated

scl Found 0 new violated cliques with the star-clique method
scl The largest star size was 4 (threshold 12)
scl Enumeration 6 times, found 0 maxl cliques
scl Greedy 0 times, found 0 maxl cliques
scl Skipped a star b/c of small solution value 0 times
scl    all cliques have been enumerated

• (1) Is there any way to switch off this output too?
• (2) Side-question: are LiftProjectCuts working correctly when the output looks like that? I'm asking because the result is correct.

Thanks Sascha

[tkralphs]

If you could give a few more details, I can try to take a look. For hooking up Cbc to cvxpy, I would think something like yaposib would be a better option (and you would get access to any other solver with an OsI interface for free).

[sschnug]

Noisy output

What details do you need? Some cuts just produce noisy output (some do not), while loglevel=0. Anything i should try?

cylp vs. yaposib

I first implemented a prototype using yaposib (only using Clp), but there are some things which lead me to cylp. Maybe some of these things are consequences about not knowing enough about the whole coinor software stack. I might add, that i am mostly interested in cbc (aka MIP-solving).

The thing which gives me the most headaches is the unmaintained state of OsiCbc (as mentioned here). Of yourse yaposib is using this as backend and i'm kind of scared regarding future developments. I'm not totally sure, if cylp manages to interface cbc without using OsiCbc (it seems cuts are incorporated through Osi), but because of the heavy use of cython i would not be surprised if that's not the case which i would prefer. Maybe you can shed some light here.

Other small stuff (again; i might be wrong / my experiences):

• cylp is much easier to install (on a very generic debian/ubuntu based system)
• cylp gives access to clg cut generators for free
• cylp seems to be more active regarding development
• cylp got a paper which explains some internals (kind of doc-bonus)
• cylp seems to have a bigger user-base

Feel free to convince me, that i am wrong.

[tkralphs]

@sschnug For details, if you could just provide a precise list of commands commands to replicate the behavior, that would be helpful.

As far as your assessment of CyLP versus yaposib, you are correct for the most part. Here are some comments.

• You're correct CyLP is not interfacing to Cbc through OsiCbc. It is directly calling methods in CbcModel. In principle, it should be easy to make calls to OsiCbc calls behave in exactly the same as the calls CyLP is making directly, but that is probably not the case at the moment. It's been on my TODO list for a long time to try to address this, since I would also like to be able to call Cbc through OsiCbc for basically the same reason as you might want to.
• With that said, I have the feeling that if you actually want maximum performance out of Cbc as a black box, the way CyLP is calling Cbc is probably not very ideal either. I think it's bypassing a lot of the "special sauce", such as auto-tuning of parameters, that OsiCbc is also bypassing. It's starting with a plain vanilla set of parameters so that the user can play with things like adding cuts, but this makes it into a little more of a "sandbox" and sacrifices performance. In general, if you are just interested in using Cbc as a black box, you want the selection of which classes of cuts to add to be be done automatically. Again, though, I have to look at this in a bit more detail to say what's going on for sure.
• CyLP does seem to have a bigger user base. At the moment, though, there aren't really any active developers of either of them. The developers of each have moved on to roles that don't allow them to contribute much and I also have a lot of other codes I'm maintaining or helping to maintain, so I have limited time to spend on these (and also don't know them that well, since I was not involved in the original development).
• It's true that CyLP seems easier to install in general, although the setup scripts are very similar in terms of how they link to COIN. I think yaposib should build pretty straightforwardly on Unbunu/Debian. The biggest difference is that yaposib has an unfortunate dependency on Boost Python, which is a pain in Windows, but easy on Linux.

You might be right that for now, CyLP seems like a good option, but both have their drawbacks and I wouldn't write off yaposib. With a few improvements to OsiCbc, it might be equally good, if not better. Certainly, you would have a more full-featured direct API with yaposib.

[sschnug]

The simple CyCbcModel from the docs is enough to observe the noisy behaviour if we add a CliqueCut-generator, like this:

# this is the basic example from the docs
# ---------------------------------------
import numpy as np
from cylp.cy import CyCbcModel, CyClpSimplex
from cylp.py.modeling.CyLPModel import CyLPModel, CyLPArray

model = CyLPModel()
x = model.addVariable('x', 3, isInt=True)
y = model.addVariable('y', 2)
A = np.matrix([[1., 2., 0],[1., 0, 1.]])
B = np.matrix([[1., 0, 0], [0, 0, 1.]])
D = np.matrix([[1., 2.],[0, 1]])
a = CyLPArray([5, 2.5])
b = CyLPArray([4.2, 3])
x_u= CyLPArray([2., 3.5])
model += A*x <= a
model += 2 <= B * x + D * y <= b
model += y >= 0
model += 1.1 <= x[1:3] <= x_u
c = CyLPArray([1., -2., 3.])
model.objective = c * x + 2 * y.sum()
s = CyClpSimplex(model)
cbcModel = s.getCbcModel()

# now add a cut-generator
# -----------------------
from cylp.cy.CyCgl import CyCglClique
cli = CyCglClique()
cbcModel.addCutGenerator(cli, name="Clique")

# set log-level of cbcModel
cbcModel.logLevel = 0
# solve
cbcModel.branchAndBound()

The following output is produced:

rcl Found 0 new violated cliques with the row-clique method
rcl The largest admissible number was 0 (threshold 12)
rcl    all row cliques have been enumerated

I didn't find any special logLevel setting for the cut-generators in the code. Maybe there is, maybe this functionality is missing.

And by the way: thanks for all the information in the previous post!