Optimizers included in the pyoptsparse not working

[yonghoonlee]

Description

Optimizers (SLSQP, etc.) included in the pyoptsparse are not working. When I install pyoptsparse separately (from official Michigan MDOLab's version) all the solvers work correctly. However, pyoptsparse included in the WEIS does not work.

Steps to reproduce issue

1. Install WEIS develop branch by running ```python setup.py develop'''
2. Run examples included in the /pyoptsparse/examples and none of them works. Since SLSQP is included by default, I primarily tested with SLSQP.
3. Note, the purpose of testing pyoptsparse included in the WEIS is to include IPOPT solver using the pyoptsparse. Currently, UIUC's development of benchmark cases heavily rely on IPOPT. Since WEIS already includes pyoptsparse, I am hoping that we can use the IPOPT through the pyoptsparse by adding IPOPT capability. However, it seems that pyoptsparse itself is currently not working well.

Current behavior

It fails to run compiled optimizers.

Expected behavior

It needs to run compiled optimizers.

[gbarter]

@johnjasa Do you have experience using IPOPT via pyOptSparse? I believe it requires an external install and I have never used it.

@yonghoonlee Can you provide some more detail about the errors you are seeing?

[yonghoonlee]

@gbarter In the WEIS folder,

cd pyoptsparse/examples
python hs015.py

output:

+------------------------------------------------------------------------------+
| pyOptSparse Error: There was an error importing the compiled slsqp module    |
+------------------------------------------------------------------------------+
Traceback (most recent call last):
  File "hs015.py", line 68, in <module>
    opt = OPT(args.opt, options=optOptions)
  File "/Users/yonghoonlee/WEIS/pyoptsparse/pyoptsparse/pyOpt_optimizer.py", line 900, in OPT
    return opt(*args, **kwargs)
  File "/Users/yonghoonlee/WEIS/pyoptsparse/pyoptsparse/pySLSQP/pySLSQP.py", line 76, in __init__
    raise Error('There was an error importing the compiled \
pyoptsparse.pyOpt_error.Error

I tried other optimizers, and they all did not work.

There is a pyoptsparse installation script prepared for OpenMDAO users ( https://github.com/OpenMDAO/build_pyoptsparse/blob/master/build_pyoptsparse.sh ), and this script installs IPOPT along with pyoptsparse. When I install this version of pyoptsparse, all other solvers work well, including SLSQP.

[yonghoonlee]

@gbarter I compared two versions of pyoptsparse source code using diff command (WEIS-included version vs MDOLab's version). They are not identical. I am not sure if the pyoptsparse included in the WEIS is either just an older version or a customized version for WISDEM or WEIS.

[yonghoonlee]

@gbarter Also FYI, when building WEIS, there was no error occurred. All the compilation went smoothly. I also tried to compile pyoptsparse separately as well, via:

cd pyoptsparse
python -m pip install -e .

and there was no error message. Building process went smoothly. However, still seeing the same error when I run some optimizer via pyoptsparse.

[gbarter]

@yonghoonlee : We have had trouble with pyOptSparse in editable/develop mode (-e). Can you try with just python -m pip install .?

[gbarter]

I am able to recreate the error. It could be that the older installation methods used in pyoptsparse are now incompatible with the recent changes to setuptools and numpy.distutils. I will take a look at the MDOLab version. We used a slightly older version of the code for compatibility with an older version of SNOPT that we have in-house.

[AthulKrishnaSundarrajan]

Hi @yonghoonlee this is an issue we faced too, when interfacing dtqpy with ipopt through pyoptsparse. We had to install pyoptsparse from conda.

conda install -c conda-forge pyoptsparse

This is how we were able to get it to work. https://mdolab-pyoptsparse.readthedocs-hosted.com/en/latest/install.html

Note that the version of pyoptsparse installed using the link above will be 2.6.x, while the version installed using WEIS will be 1.0.x

[yonghoonlee]

@AthulKrishnaSundarrajan thank you for your suggestion. I also circumvented the issue with custom install using OpenMDAO team’s script, but did not try conda-forge. I will try this and see if everything works well.

[johnjasa]

Hi all, thanks for the great discussion here. I think @gbarter's PR addresses these issues well, but I will weigh in for completeness.

If you need IPOPT, conda installing pyOptSparse is the best method. However, SNOPT requires pyOptSparse to be built from source. To get both, you can potentially: 1) install IPOPT separately and point pyOptSparse to its location during compilation, or 2) install pyOptSparse via conda then drop the SNOPT source in there and recompile. I myself haven't gotten either method to work, but haven't spent a considerable amount of time on it.

If you don't have workarounds or can't access the optimizers you need, let me know and I can dig more into getting both installed correctly.

I'm looping in Neil Wu at UMich (@nwu63) only because he previously gave me some tips and I want to credit him for them. Specifically, he said this:

""" I haven't tried [installing both IPOPT and SNOPT] yet, but looking at the conda recipe here, in particular the build script it seems that if you

1. install conda via IPOPT
2. set the $IPOPTDIR correctly (this will depend on your conda environment setup), but probably accessible via some $CONDA environment variable
3. put SNOPT source code as you would normally
4. run "pip install ."

That should work I think. """

[gbarter]

This resource might also be helpful: https://github.com/OpenMDAO/build_pyoptsparse

[gbarter]

For the NREL crew, here is a fork of pyoptsparse that should compile nicely and use our older version of SNOPT: https://github.com/WISDEM/pyoptsparse

[gbarter]

Fixed in #160 PR