search

OpenMDAO / OpenMDAO

OpenMDAO repository.
http://openmdao.org
Other
551 stars 251 forks source link

Properly handle AnalysisError from ScipyOptimizer #2530

Closed JustinSGray closed 2 years ago

JustinSGray commented 2 years ago

Issue Type

Description

PyOptSparseOptimizer has code to catch AnalysisErrors and a) report failure to optimizer b) record the failed case and keep going.

Scipy optimizer driver should, if possible, have the same abilities

Example

Note: missing features pointed out in SO question here: https://stackoverflow.com/questions/66480532/in-openmdao-is-there-a-way-to-ensure-that-the-constraints-are-respected-before/66494488?noredirect=1#comment127742263_66494488

swryan commented 2 years ago

Unfortunately, SciPy optimize does not provide the functionality of handling a failure from the objective/gradient functions the way that pyoptsparse does (i.e. we can't a) report failure to optimizer).

The best we could do is ignore AnalysisErrors (or throw them as warnings) and hope to "pass through this bad region" (i.e. b) record the failed case and keep going).

@Kenneth-T-Moore

Kenneth-T-Moore commented 2 years ago

Since a) is probably not possible, do we still even want b)? We probably wouldn't always want to "record and keep going" after a raised AnalysisError, if the error is something that you can fix or correct. Do we need a switch to direct how the scipy optimizer handles AnalysisErrors?

Also, it isn't clear what you return for the objective and constraints -- repeat the previous successful values? Seems like the optimizer could end up stuck at this point anyway.

I will agree though that ScipyOptimizer is the oddball. It is our only driver can't handle an AnalysisError. Not sure what to do about that.

robfalck commented 2 years ago

Does this work with pyOptSparse/SLSQP? I know there's talk there about redoing their SLSQP implementation, perhaps this should be taken into consideration. Has this capability been tested with IPOPT, I know their API supports it but I don't think I've tried it yet.

As far as supporting ScipyOptimizeDriver, I agree it doesn't seem like we have the capability. I think silently failing would be a bad idea. I wish we could support the drivers more equally and pyOptSparse was not the defacto optimizer for OpenMDAO, but short of changing code on the Scipy side I just don't see how we do this.

andrewellis55 commented 2 years ago

Just to chime in, how receptive might SciPy be to a PR to enable this within their code? It seems like it would make their codebase more robust as well. Or at the least an issue raised to suggest this within their code.

swryan commented 2 years ago

From a cursory look at the pyoptsparse code, neither pySLSQP nor pyIPOPT do anything with the failure flag returned from the obj/grad functions, in contrast with pySNOPT, which does return that information in the form of mode to snopt.

robfalck commented 2 years ago

@andrewellis55 In my experience they may be receptive to it but it would take a while to implement. I'm going to close this for now and we can revisit it in the future. We'd likely have more luck implementing this in IPOPT, for those who don't have access to SNOPT.