alternative optimization package?

[ryancoe]

We are currently using scipy.optimize.minimize.

We have previously played with using IPOPT for the pseudo-spectral solution. From what I can tell, this is developed using C++ by COIN-OR; there are a couple of Python wrappers.

[ryancoe]

It looks like this conda package might be an option: https://anaconda.org/conda-forge/ipopt

EDIT: this is just the binary, not a Python wrapper

[cmichelenstrofer]

I was able solve the optimization problem with either scipy.optimize.minimize or ipopt via the cyipopt wrapper. The optimization package was not the issue (see #10). Since scipy does just fine, we will stick with it for now. If we want we can add a flag to switch between optimizers in the future.

[ryancoe]

Re-opening this as we really should take another look at IPOPT. I believe the best option for using this in Python is cyipopt

[cmichelenstrofer]

Pyomo is a Sandia developed code (moved from internal to public GitHub in 2016) that is becoming very popular (actively maintained/developed, weekly developer cals open to the community, thousands of users, some big projects using it). I have been meaning to check it out. Not sure if it is a good option for this project.

• Github
• Website

[rebeccamccabe]

I am wondering if you have considered pyoptsparse - it contains wrappers for IPOPT, SNOPT, and various other solvers with a single interface, and is the primary optimization driver for OpenMDAO, which I plan to use in my project. (Separately, I'm still figuring out a strategy for using WOT with OpenMDAO because the nested optimization structure makes it a "collaborative optimization" which requires a different setup.) But perhaps pyoptsparse would be helpful for WOT in the future, while also potentially facilitating OpenMDAO integration.

[cmichelenstrofer]

@rebeccamccabe I wasn't aware of it but seems like a good option!

[ryancoe]

@rebeccamccabe - Thanks very much for this suggestion! Briefly looking into this a bit more... pyoptsparse does look interesting, but digging a bit deeper it looks like some of the methods we're immediately interested in using (e.g., IPOPT) require separate installation, often via compiling on your machine. This is definitely something that would discourage us from using the package, as our average user would likely be challenged to get the install done right. Maybe it would work fine if we identify a method within pyoptsparse that works well for our problem and does not require a complex installation. I'd definitely like to see if there's a better option out there for us, so please keep the suggestions coming.

[ryancoe]

I was doing some reading tonight on a related topic and looked again at options for IPOPT Python interfaces. It looks like cyipopt has been under very steady development for a while now and has a scipy-style interface. I was able to install it and run the examples. I was even able to install the CPU jaxlib and run an example where that's used to provide the jacobian! I think this might be a winner. @cmichelenstrofer @michaelcdevin - Any thoughts?

[cmichelenstrofer]

sounds good to me! Would we gain any new capability?

[gbacelli]

yes, it comes with chatGPT... 😀

[ryancoe]

Would we gain any new capability?

@cmichelenstrofer - Not entirely sure... when @gbacelli and I were planning the restructuring of WecOptTool (when we transitioned from MATLAB to Python) we got some advice from an optimization expert. I remember two things in particular:

1. Use some sort of autodiff (done)
2. Consider IPOPT

It'd probably be best to find some problem(s) where we struggle (e.g., the convergence is sensitive to the scaling factors) and swap cyipopt for scipy to see if there's any improvement. Now that the installation for cyipopt is straightforward and it uses the same structured interface as scipy, I think this should be a relatively quick task.

Critical context: I believe that when we call scipy.optimize.minimize, it's using the SQLP method.

[gbacelli]

FYI: Interior point methods (the ones implemented in IPOPT) are the preferred solutions for numerical optimal control problems.

[michaelcdevin]

cyiopt looks great, good find @ryancoe!

My only hesitation is that cyiopt has an EPL-2.0 license, which has a derivative works clause. WecOptTool is currently licensed under GPL-3.0. However, I believe EPL is less restrictive than GPL with what constitutes a "derivative work"; I think if we use cyiopt without modification, we can continue using a GPL-3.0 license without issue. We should verify this before making any changes though.

[rebeccamccabe]

@rebeccamccabe - Thanks very much for this suggestion! Briefly looking into this a bit more... pyoptsparse does look interesting, but digging a bit deeper it looks like some of the methods we're immediately interested in using (e.g., IPOPT) require separate installation, often via compiling on your machine. This is definitely something that would discourage us from using the package, as our average user would likely be challenged to get the install done right. Maybe it would work fine if we identify a method within pyoptsparse that works well for our problem and does not require a complex installation. I'd definitely like to see if there's a better option out there for us, so please keep the suggestions coming.

There is a provided script to handle the build and install of ipopt for you in pyoptsparse https://github.com/OpenMDAO/build_pyoptsparse/. Some of the pyoptsparse features might be handy like parallelism, easily swapping solvers, history, and open MDAO driver, though I don’t see a problem with cyipopt either. Pyoptsparse having an open MDAO driver doesn’t matter nearly as much as I thought it would because “At this time, OpenMDAO does not evaluate gradients through the optimization process, so a gradient-based driver at the outer level cannot access derivatives across the optimization on the inner level.“ So it would be helpful if you wanted to run WecOptTool via open mdao in the future, ie make subsystems for hydro and geometry and PTO, but not helpful for a nested optimization (which is likely the structure of my project). (https://openmdao.org/newdocs/versions/latest/features/building_blocks/components/submodel_comp.html). Pyoptsparse doesn’t have auto diff, though open mdao has some jax integration.

[cmichelenstrofer]

My only hesitation is that cyiopt has an EPL-2.0 license, which has a derivative works clause. WecOptTool is currently licensed under GPL-3.0. However, I believe EPL is less restrictive than GPL with what constitutes a "derivative work"; I think if we use cyiopt without modification, we can continue using a GPL-3.0 license without issue. We should verify this before making any changes though.

We do not include their code as part of our source code/repo... it is ambiguous whether using a package through Python import constitutes derivative work... but in any case I opened an issue asking them to list GPL as a secondary license (EPL's mechanism to allow use by software under different licenses).

[cmichelenstrofer]

When making the changes to cyiopt + jax we should think about if we can create a mechanism for easily changing the optimization package. I think it would only require allowing to overload a small part of the WEC.solve function. This would "future proof" the solve step and allow other workflows (e.g. #272).

[moorepants]

We do not include their code as part of our source code/repo... it is ambiguous whether using a package through Python import constitutes derivative work

I think you are safe with this method, as you are not distributing cyipopt. If you distribute IPOPT in a GPL codebase, then you run foul of the incompatibility.