search

hpc4cmb / toast

Time Ordered Astrophysics Scalable Tools
Other
44 stars 39 forks source link

Does TOAST need `MPI_THREAD_MULTIPLE`? #678

Closed ickc closed 1 year ago

ickc commented 1 year ago

When trying to run TOAST 2 or 3 tests with OpenMPI 4, I'm experiencing exactly the same error in https://github.com/mpi4py/mpi4py/issues/34

They recommend setting

import mpi4py
mpi4py.rc.thread_level = 'single' # or perhaps 'serialized'

in the beginning if MPI_THREAD_MULTIPLE is not needed. Does TOAST need it, and if not, could we set that in TOAST when importing mpi4py?

Relevant doc:

tskisner commented 1 year ago

All MPI operations in toast happen outside of threaded regions and in python. So, single should be fine. I believe the other (optional) MPI-enabled external packages we can use (e.g. libmadam and libconviqt) also follow this pattern. The only place in toast that mpi4py is optionally imported is in toast/mpi.py, so feel free to open a PR setting that rc parameter.