Pyomo / pyomo

An object-oriented algebraic modeling language in Python for structured optimization problems.
https://www.pyomo.org
Other
1.98k stars 509 forks source link

Incomplete optimisation data with CPLEX solver #1811

Open ghost opened 3 years ago

ghost commented 3 years ago

Hi,

I've been trying out Pyomo for the past few days and am very pleased with it. Today, however, I noticed I am unable to retrieve all the optimisation data that I would normally be able to obtain from CPLEX (via the command line, matlab, python bindings, etc).

In particular, I'm referring to the best bound, final relative MIP gap, deterministic computation time, and the feasible solution pool.

I suppose I could get this information by writing the problem to a file and doing what I need to do externally (without using Pyomo for those steps). The problem with this approach is that it would require me to develop two closely-related scripts (one in Pyomo and one using some other technology) to know which variable is which, therefore limiting Pyomo's usefulness.

Alternatively, I could get that information from the cplex object (used in the official cplex python scripts) since I'm using cplex_direct, but I can't seem to find that object.

This issue has also been identified in: https://groups.google.com/g/pyomo-forum/c/61VcjzUrzds

Is there a simpler way to access this information? If not, are there plans to complement the information provided through the solve method?

Any help is much appreciated.

michaelbynum commented 3 years ago

I believe the best bound and wall clock time are loaded into the results object with the cplex_direct interface. If you want to access the underlying cplex object, you will have to use the cplex_persistent interface. We currently only have one solver interface that supports solution pools (the gurobi_persistent interface), but it should be easy to add to the cplex_persistent interface.

ghost commented 3 years ago

Thank you michaelbynum.

I haven't tried using the cplex_persistent interface but I will soon.

Are the 'Lower bound' and 'Higher bound' fields in the solution object the best bound and incumbent solution values (respectively, and vice-versa, depending on the optimisation sense)? They are always the same but I've only tested very simple problems for which CPLEX and GLPK always find an optimal solution (i.e., there would be no difference anyway). I plan on testing more complicated problems in the coming days to check this.

On a more general note, where can I find detailed information about the solution objects other than by going through Pyomo's source code? They seem to be different depending on the solver, which makes sense to some extent, but I can't seem to find more information about them. It would be good to have this documented for those of us who are not Python experts, if it isn't already.

michaelbynum commented 3 years ago

Yes, results.problem.lower_bound and results.problem.upper_bound provide the best bound and best incumbent, depending on objective sense.

We are currently working on cleaning up the solver interfaces and results objects, but it is taking some time.

jsiirola commented 3 years ago

@michaelbynum, I was recently digging into the CPLEX.py interface, and it appears that the parser is actually setting the lower_bound == upper_bound in the results object any time that CPLEX returns "optimal":

https://github.com/Pyomo/pyomo/blob/13cd4f8ebe3e641fc76278e0280e2cca2ce8786d/pyomo/solvers/plugins/solvers/CPLEX.py#L810-L818

This strikes me as a bug in the LP (and maybe NL) interface.

michaelbynum commented 3 years ago

I agree that this is a bug. I believe the direct and persistent interfaces handle this correctly, but better testing of this is definitely warranted.

bknueven commented 3 years ago

I agree with @michaelbynum that the cplex direct/persistent interfaces are doing the right thing with regards to bounds. See: https://www.ibm.com/support/knowledgecenter/SSSA5P_12.8.0/ilog.odms.cplex.help/refpythoncplex/html/cplex._internal._subinterfaces.MIPSolutionInterface-class.html https://github.com/Pyomo/pyomo/blob/40187a85475d8a72397fc99019c7aa575914582d/pyomo/solvers/plugins/solvers/cplex_direct.py#L695-L705