search

Pyomo / pyomo

An object-oriented algebraic modeling language in Python for structured optimization problems.
https://www.pyomo.org
Other
2.01k stars 516 forks source link

Fast(er) direct solver interfaces #1160

Closed whart222 closed 4 years ago

whart222 commented 4 years ago

I think users would generally expect direct solver interfaces to be faster, since we're avoiding file I/O. However, I think this is not always the case. For example, see #515

We need to critique our direct solver interfaces and assess whether we can setup/analyze problems more effectively. This activity could involve:

  1. Identifying bottlenecks in existing solver interfaces
  2. Exploring strategies for avoiding canonicalization in Pyomo (e.g. directly generating solver expressions from Pyomo expression trees)
  3. Interfacing with POEK/COEK to see if their interfaces can resolve known bottlenecks.

NOTE: Aside from #51 and #1169, I only have anecdotal evidence that direct solver interfaces aren't faster. I would recommend confirming that they are faster before closing this issue.

emma58 commented 4 years ago

I'm not sure whether @michaelbynum's recent enhancements change this, but the other thing that can make the persistent interface slower is hacking around functionality that's not yet in the persistent interface. The obvious one is changing the right-hand side of a constraint: The only way I know how to do this now is to remove and add the constraint, but I've never found this strategy to be worth the time. I just point this out because I wonder if there aren't other cases where the potential performance enhancements might be in what is not there rather than what is.

qtothec commented 4 years ago

I have been talking to Steve about making the GAMS direct "solver" faster by constructing in-memory GMO objects rather than writing a file, as well.

qtothec commented 4 years ago

As a follow-up to this, Steve Dirkse sent me an example for building up and solving an NLP using in-memory GMO and GEV.

carldlaird commented 4 years ago

I am not sure that #515 is the correct issue to reference. Does anyone know the issue number that shows a case where the direct solvers are slower than the indirect solvers?

qtothec commented 4 years ago

Well, for the GAMS case, I have this: https://github.com/Pyomo/pyomo/pull/215#issuecomment-328152064

I don't know about the MIP solver cases.

whart222 commented 4 years ago

@carldlaird I updated the references to related issues. Both of these concern the CPLEX python solver interface.

jsiirola commented 4 years ago

Archived on the master Performance Proposals Issue (#1430). Closing this performance proposal until active development has begun.

staadecker commented 2 years ago

Fyi I know that for the large models I'm working with the direct gurobi solver interfaces is slower than the IO gurobi solver interface