Provide a way to let a user tell pyoptsparse to NOT use mpi4py

[hschilling]

Type of issue

What types of issue is it? Select the appropriate type(s) that describe this issue

• New feature (non-breaking change which adds functionality)

Description

Provide a way to let a user tell pyoptsparse to NOT use mpi4py.

I work on the OpenMDAO team and it has happened that sometimes we can totally break our MPI installation on our dev machines. It's handy to tell OpenMDAO to skip importing mpi4py if just want to run some code that doesn't make use of MPI while you work on fixing the MPI install.

So there is an environment variable, OPENMDAO_REQUIRE_MPI, that can be set to '0'. (Really anything other than ['always', '1', 'true', 'yes']) and then OpenMDAO skips importing mpi4py.

The problem is that we also obviously import pyoptsparse and it imports mpi4py no matter what. We have no control over that.

I see two places in the pyoptsparse code where code would need to be added to give the user control:

• At the bottom of pyOpt_MPI.py
• At the top of pyParOpt/ParOpt.py

Current behavior

When using pyoptsparse, there is no way to force pyoptsparse to skip importing mpi4py.

Expected behavior

Provide some environment variable ( or some other mechanism ) for the user to set so that mpi4py is not imported.

Code version (if relevant)

NA

[ewu63]

Thanks for writing this up, I can see why you guys want this feature. In my mind this is relatively straightforward to implement -- wrap the MPI imports with a conditional based on some environment variable. We could define something like PYOPTSPARSE_REQUIRE_MPI for this purpose (and then document this behaviour of course). Note that ParOpt requires MPI, so it will have to be disabled if mpi4py is not used.

Would you/others on the OpenMDAO dev team be willing to do this and then make a PR?

[hschilling]

Yes, I would be happy to. FYI, here is an example of how we do it in OpenMDAO:

# If OPENMDAO_REQUIRE_MPI is set to a recognized positive value, attempt import
# and raise exception on failure. If set to anything else, no import is attempted.
if 'OPENMDAO_REQUIRE_MPI' in os.environ:
    if os.environ['OPENMDAO_REQUIRE_MPI'].lower() in ['always', '1', 'true', 'yes']:
        from mpi4py import MPI
    else:
        MPI = None
# If OPENMDAO_REQUIRE_MPI is unset, attempt to import mpi4py, but continue on failure
# with a notification.
else:
    try:
        from mpi4py import MPI
    except ImportError:
        MPI = None
        sys.stdout.write("Unable to import mpi4py. Parallel processing unavailable.\n")
        sys.stdout.flush()
    else:
        # If the import succeeded, but it doesn't look like a parallel
        # run was intended, don't use MPI
        if MPI.COMM_WORLD.size == 1:
            MPI = None

[hschilling]

I have the code ready but can't find a good place to document it. Suggestions?

[ewu63]

That's a good point, I see that we have no documentation on anything parallel. I suppose for now we can add the info to the Install page, under requirements. Could just mention something along the line of

pyOptSparse can optionally run in parallel if a suitable mpi4py installation exists. This will be automatically detected and imported at run time. If you explicitly do not wish to use mpi4py for serial runs, ____

Would that work? I know we are significantly lacking in documentation, so any suggestions are welcome -- potentially in the future even a dedicated page for integration with OpenMDAO perhaps?

[hschilling]

Works for me ! I will add it and PR it later today, I hope. Thanks

An OpenMDAO page would be a good idea! I could talk to the team about that.

[ewu63]

Great, thanks a lot for doing this work.