mpiexec is not working with pyGeo imported

[stankarpuktubs]

Description

Parallel computation using mpiexec is not running if pyGeo is imported

Steps to reproduce the issue

1. Create a sample file for parallel executions:

   hello.py

   from mpi4py import MPI comm = MPI.COMM_WORLD rank = comm.Get_rank() print("hello world from process ", rank)

2. Then, create a file in which the mpiexec is executed and import pyGeo there : from pygeo import pyGeo def run_aerodynamic_analysis(Input): with open('SU2_output.log', 'w') as f: p = subprocess.call(['mpiexec', '-n',str(5), 'python','hello.py'], stdout= f, stderr= None, stdin=subprocess.PIPE) print(p)

3. Try to run with the module imported and not imported

Current behavior

The code does not run the parallel computation and returns p=1

Expected behavior

The parallel computation shall run and give no error

Code versions

• Operating System:
• Python: 3.8.10
• OpenMPI:
• CGNS:
• PETSc:
• Compiler:
• This repository:

[hajdik]

@A-CGray and I tested locally and determined that importing pyGeo (or several other MDO Lab codes) does in fact prevent any subprocesses that are launched in that script from running. We're pretty sure this is because our codes use mpi4py, so that is already imported when importing any of our packages. To see this, you can import mpi4py in the top level script instead of pyGeo and the same failure happens.

Subprocesses that call MPI when MPI has already been imported seem to not work, there might be a way for you to pass in the current MPI context in your top level script. I don't think we can address this on our end as pyGeo and our other packages are set up to be called from parallel runscripts as in the MACH-Aero tutorial.

If you can't set up a parallel runscipt, you can import pyGeo after the subprocess call in the top level script and get the expected behavior.

[stankarpuktubs]

Thank you very much for your quick answer! Unfortunately, I need to run SU2 after the pygeo since I am exporting the pygeo wing into Pointwise, mesh it and then run with a CFD code. I will try to find a solution but I am at least sure where my code is not working and what the issue is. Thank you again!

[ewu63]

Just to chime here, importing mpi4py will essentially call MPI_Init, see here, and this will likely interfere with spawning an MPI subprocess. I haven't tried this but maybe you can call MPI.Finalize() after all pygeo operations and before you call mpiexec via subprocess? An alternative would be to split into separate subprocesses, e.g. a bash script that calls pygeo first, then CFD such that MPI environments do not interfere.

[stankarpuktubs]

Indeed the solution with MPI.Finalize() worked perfectly. Now, I can run the subprocess. Thank you very much!