Example in PyROS docs doesn't use solver of the correct class

[makansij]

The example taken from the docs https://pyomo.readthedocs.io/en/stable/contributed_packages/pyros.html?highlight=pyros#step-3-solve-with-pyros

Does not work.

`
# === Designate which variables correspond to first- and second-stage degrees of freedom ===
first_stage_variables =[m.x1, m.x2, m.x3, m.x4, m.x5, m.x6,
                     m.x19, m.x20, m.x21, m.x22, m.x23, m.x24, m.x31]
second_stage_variables = []
# The remaining variables are implicitly designated to be state variables

# === Call PyROS to solve the robust optimization problem ===
results_1 = pyros_solver.solve(model = m,
                                 first_stage_variables = first_stage_variables,
                                 second_stage_variables = second_stage_variables,
                                 uncertain_params = uncertain_parameters,
                                 uncertainty_set = box_uncertainty_set,
                                 local_solver = pe.SolverFactory('baron'),
                                 global_solver= pe.SolverFactory('baron'),
                                 options = {
                                    "objective_focus": pyros.ObjectiveType.worst_case,
                                    "solve_master_globally": True,
                                    "load_solution":False
                                  })
# ===========================================================================================
# PyROS: Pyomo Robust Optimization Solver ...
# ===========================================================================================

# INFO: Robust optimal solution identified. Exiting PyROS.

# === Query results ===
time = results_1.time
iterations = results_1.iterations
termination_condition = results_1.pyros_termination_condition
objective = results_1.final_objective_value
# === Print some results ===
single_stage_final_objective = round(objective,-1)
print("Final objective value: %s" % single_stage_final_objective)
# Final objective value: 48367380.0
print("PyROS termination condition: %s" % termination_condition)
# PyROS termination condition: pyrosTerminationCondition.robust_optimal

However, if you change

                                 global_solver= pe.SolverFactory('baron'),

to be

                                 local_solver = pe.SolverFactory('appsi_ipopt'),
                                 global_solver= pe.SolverFactory('appsi_ipopt'),

Then it works.

[makansij]

Closed by accident. Sorry - I was intending to edit my post!

[blnicho]

@makansij could you post the error you're seeing and what versions of Python and Pyomo you're using?

[makansij]

Certainly. Here's the error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-26-78116d21f8ab> in <module>
     16                                     "objective_focus": pyros.ObjectiveType.worst_case,
     17                                     "solve_master_globally": True,
---> 18                                     "load_solution":False
     19                                   })
     20 # ===========================================================================================

/usr/local/lib/python3.7/site-packages/pyomo/contrib/pyros/pyros.py in solve(self, model, first_stage_variables, second_stage_variables, uncertain_params, uncertainty_set, local_solver, global_solver, **kwds)
    326 
    327         # === Validate kwarg inputs
--> 328         validate_kwarg_inputs(model, config)
    329 
    330         # === Validate ability of grcs RO solver to handle this model

/usr/local/lib/python3.7/site-packages/pyomo/contrib/pyros/util.py in validate_kwarg_inputs(model, config)
    426     # === Solvers provided check
    427     if not config.local_solver or not config.global_solver:
--> 428         raise ValueError("User must designate both a local and global optimization solver via the local_solver"
    429                          " and global_solver options.")
    430 

ValueError: User must designate both a local and global optimization solver via the local_solver and global_solver options.

Version of pyomo: 6.4.1 Version of Python: 3.7

I could try to force the module to be imported under python 3.9. Currently it just imports automatically from /usr/local/lib/python3.7/site-packages/pyomo/__init__.py, but I also have pyomo under /Users/<username>/opt/anaconda3/lib/python3.9/site-packages.

[makansij]

Also, I do seem to have the "baron" solver installed, as shown by this output:

pyo.SolverFactory('baron')
>>> <pyomo.solvers.plugins.solvers.BARON.BARONSHELL at 0x10b020810>

[michaelbynum]

@makansij, Can you check pyo.SolverFactory('baron').available()?

[makansij]

pyo.SolverFactory('baron').available() yields False!

[michaelbynum]

We should improve the error message on line 428 of pyros/util.py

[makansij]

How do I make baron available to pyomo?

I tried giving it the path explicitly:

import pyomo
import pyomo.environ as pe
solvername = 'baron'
solverpath_exe = '<path-to-baron-directory>/baron-osx64/baron'
solver = pe.SolverFactory(solvername, executable=solverpath_exe)

However, pe.SolverFactory(solvername).available() still yields False.

For all of the other solvers, there's a <solver>.py file in pyomo/contrib/appsi/solvers directory. What is the equivalent for the baron solver?

[makansij]

@michaelbynum What should I do to resolve this? And - how can I help the Pyomo/PyROS community? Thanks.

[jsiirola]

@makansij, Pyomo's BARON interface works by looking on the system PATH for the "baron" executable. available() reports False when it could not find the executable. As you point out, you can create an instance of the BARON solver interface that points to a specific executable by passing the executable= option to the SolverFactory call. However, that does not change the system PATH, so subsequent SolverFactory('baron') calls will generate BARON interfaces that still cannot find the solver executable.

The solution is either one of the following:

1. add the '<path-to-baron-directory>/baron-osx64 directory to your PATH environment variable
2. create the Baron solver interface and explicitly specify the path to the solver, and pass that solver object to PyROS to use; e.g.,

   solverpath_exe = '<path-to-baron-directory>/baron-osx64/baron'
   baron_solver = pe.SolverFactory('baron', executable=solverpath_exe)
   # === Call PyROS to solve the robust optimization problem ===
   results_1 = pyros_solver.solve(model = m,
   # ....
   local_solver = pe.SolverFactory('ipopt'),
   global_solver= baron_solver,
   # ....
   })

Addressing your other question: the standard Pyomo solver interfaces are all in pyomo/solvers/plugins/solvers. The BARON interface is BARON.py. APPSI is a new set of experimental solver interfaces that are currently under development (this is why they are in contrib).

[makansij]

@jsiirola Thanks that was very helpful!

Now, I'm able to get PyROS to attempt to solve the optimization problem. However, the solver exits with ValueError: Cannot load a SolverResults object with bad status: aborted. It only gets to iteration 0 :

INFO: PyROS working on iteration 0...

ValueError                                Traceback (most recent call last)
<ipython-input-9-b7058cce1f86> in <module>

Again, this is copied exactly from https://pyomo.readthedocs.io/en/stable/contributed_packages/pyros.html#pyros-usage-example

If you think this is the appropriate place to work through bugs, we can continue this discussion here. But if you recommend another way to solve this, please let me know.

[jsiirola]

@makansij this is probably the best place to continue the discussion. It looks like the initial problem solve is failing. Just wondering: do you have a BARON license, or are you running it in “demo” mode?

[makansij]

AFAIK I don't have a license, so I'm running it in "demo" mode.

[jsiirola]

That example almost certainly exceeds the Baron demo license limits.

@shermanjasonaf, we might consider adding a more informative error message when the initial solves fail. At a minimum, recommending that users run with tee=True would let them see the error tossed by the solver.

[makansij]

Okay, gotcha. So, I can't solve this problem using the demo license. What are my other options for non-convex MINLP solvers?

I have a robust optimization problem that I want to solve, and one of the reasons PyROS is attractive to me is because it emphasizes robustly feasible solutions, even to non-convex MINLP Robust Optimization problems. What are my other options with PyROS?

Others, e.g. ROmodel, do not handle non-convex and integral uncertainty sets. Thanks.

[michaelbynum]

@makansij You are specifically looking for an open-source MINLP solver?

[makansij]

One that can integrate with PyROS, yes.

[shermanjasonaf]

PyROS has not yet been tested for two-stage robust MINLPs; per the documentation, the current implementation only provides support for models with continuous variables (robust NLPs).

If you are looking for an open-source deterministic MINLP solver, you could try COUENNE. I expect this to be compatible with PyROS in the event that support for robust MINLPs is added.

[makansij]

@shermanjasonaf Yes, my variables (I.e “x” in the problem formulation) are continuous. But, as I said before, my uncertain parameters are continuous and discrete. But I didn’t see anywhere in the docs that indicated a discrete and continuous parameter is not supported. The docs say that “q” can be discrete or continuous, and I assumed it’s okay for “q” to include both discrete and continuous variables. Using the right uncertainty set it seems like this should be possible at least in theory – right?

[shermanjasonaf]

PyROS currently supports sets for which all parameters are discrete, or all parameters are continuous; to my knowledge, sets with both discrete and continuous parameters have not yet been tested. Addressing an uncertainty set with both discrete and continuous uncertain parameters should be possible in principle---doing so will entail adding a routine which accounts for such sets during the separation phase of each PyROS iteration. We will seek to add this functionality in the future. In the meantime, if you are simply looking for a robust feasible (rather than a robust optimal) solution, then you may consider optimizing against the convex hull of the discrete dimensions as an alternative---though this may be too conservative for your model(s) of interest.

[makansij]

PyROS currently supports sets for which all parameters are discrete, or all parameters are continuous; to my knowledge, sets with both discrete and continuous parameters have not yet been tested. Addressing an uncertainty set with both discrete and continuous uncertain parameters should be possible in principle---doing so will entail adding a routine which accounts for such sets during the separation phase of each PyROS iteration. We will seek to add this functionality in the future. In the meantime, if you are simply looking for a robust feasible (rather than a robust optimal) solution, then you may consider optimizing against the convex hull of the discrete dimensions as an alternative---though this may be too conservative for your model(s) of interest.

Right. I also saw your message on the other thread - thanks @shermanjasonaf ~!

[makansij]

Thanks @shermanjasonaf for your help thus far. I tried to install COUENNE per the link that you provided. However, the page doesn't seem to be maintained anymore (no updates in 3 years), and several of the links on that page in the install instructions, are broken.

Can you suggest an easier method for install couenne on a Mac? I got started with the "Couenne" directory by downloading it from here, but it's not clear to me where to go from there. Thanks again.

[shermanjasonaf]

@makansij I currently have access to COUENNE through the IDAES Toolkit, which itself can be installed using the instructions here.

@jsiirola Do you know any simple methods for installing COUENNE (on a Mac)? Or will an IDAES installation suffice?

[makansij]

Hi @shermanjasonaf. Thanks. I followed the instructions for "MacOS" (not using docker) here https://idaes-pse.readthedocs.io/en/1.4.4/install/index.html#mac-osx, and installed COUENNE. However, when I type "whereis couenne", there is no path printed as output.

When I ran pytest --pyargs idaes -W ignore as instructed, there were a few "ERRORs", but as is suggested, these should be benign so I ignored them.

However, it's not clear to me how PyROS/Pyomo will find the couenne binaries? I ran whereis couenne and nothing is printed as output. Do I need to obtain the binaries from somewhere else? Or am I missing something else about this installation?

[shermanjasonaf]

@makansij I believe the issue is due to the note shown below step 2 of the installation instructions (binary extensions not yet supported for Mac)

[makansij]

@makansij I believe the issue is due to the note shown below step 2 of the installation instructions (binary extensions not yet supported for Mac)

Does that mean I shouldn't use the IDEAS toolkit that you referred to for installing COUENNE on a Mac?

Thanks