xESMF 8 possible without MPI?

[mathomp4]

All,

A question. I help manage a Python stack at a computing center and a user has requested xESMF. The problem is that xESMF wants to install MPI. If it doesn't, we get #47 where a lot of tests don't pass with:

ImportError: Regrid(filename) requires PIO and does not work if ESMF has not been built with MPI support

My issue is this. I really do not like or want other mpirun, or mpifort or other MPI executables and libraries in the Python stack because in the past this has led to issues in the past. E.g., people are having problems because the mpirun being used is not the one from the Intel MPI module that compiled their code but from the mpich3 python module!

And, our modulefile, while limited for our python does have:

prepend_path{"PATH",pathJoin(pkgdir,"bin")}

and in that is an mpirun. I know that careful loading can bury that in PATH behind others, but a user will eventually load python after, say, Intel MPI and boom, wrong mpirun.

Is it possible to have a recent xESMF (v8+) that doesn't need mpi? Would it work if I deleted the mpi* files/scripts from the bin/ directory? I'd like to add this package, but in a safe way.

I ask about "recent" xESMF because our model (GEOS) uses the cubed-sphere. In fact, it is our cubed-sphere grid in ESMF, thus ESMF 8 would be nice to have. (Note: we do install ESMF as part of our base libraries. Is there a way to point xesmf to that?)

[rokuingh]

Speaking from the ESMPy perspective: the ImportError that you reference is only raised if a user attempts to call Field.read() from an installation that was built without MPI. xESMF could easily supply this functionality using netCDF4, and forego the MPI requirement of ESMPy for this feature. There may be other xESMF dependencies on MPI though..

[mathomp4]

@rokuingh Well, the only mention of "mpi" in the xESMF repo is the on the installation page, so maybe there is a chance.

Now, my reading of ESMPy is that that error is from regrid.py when you try and write the weights, not read them. And from my reading of xESMF I think there is no way to call ESMPy without writing weights? (Is that right, @JiaweiZhuang?) ESMPy seems to have support for a no-write regrid (via ESMP_FieldRegridStore) if no filename is passed in, but I think xESMF always passes in a filename.

[mathomp4]

Slight update. I was able to install xesmf 8 following #47 and pytest does succeed (with warnings). I can even then delete the mpi* from bin/ and it still seems to work. Not perfect, but perhaps a workaround that might be safe on my cluster (though having mpi4py there still makes me cringe...).

Note the warnings were:

====================================================================================== warnings summary ======================================================================================
tests/test_backend.py: 1 warning
tests/test_frontend.py: 29 warnings
  /discover/nobackup/mathomp4/Miniconda-Test/python3/latest_py3.8/2020-07-30/lib/python3.8/site-packages/ESMF/interface/cbindings.py:2095: DeprecationWarning: Call to deprecated function ESMP_FieldRegridStoreFile.
    def ESMP_FieldRegridStoreFile(srcField, dstField, filename,

tests/test_frontend.py::test_existing_weights
tests/test_frontend.py::test_existing_weights
  /discover/nobackup/mathomp4/Miniconda-Test/python3/latest_py3.8/2020-07-30/lib/python3.8/site-packages/xesmf/frontend.py:245: DeprecationWarning: regridder.A is deprecated and will be removed in future versions. Use regridder.weights instead.
    warnings.warn(message, DeprecationWarning)

tests/test_frontend.py::test_regrid_dask
tests/test_frontend.py::test_regrid_dask
  /discover/nobackup/mathomp4/Miniconda-Test/python3/latest_py3.8/2020-07-30/lib/python3.8/site-packages/xesmf/smm.py:70: UserWarning: Input array is not C_CONTIGUOUS. Will affect performance.
    warnings.warn("Input array is not C_CONTIGUOUS. "

-- Docs: https://docs.pytest.org/en/stable/warnings.html
============================================================================== 42 passed, 34 warnings in 22.47s ==============================================================================

[mathomp4]

Also interesting: it looks like when xESMF does do a regridder, it only uses one core/process.

ETA: though maybe that's because I removed mpirun et al?

[jeffreysward]

Thanks for this! I can confirm that the addition of the mpi* executables in my env confused our scheduler, but the simple deletion of them from bin/ after installing xESMF also worked for me.

[mathomp4]

Thanks for this! I can confirm that the addition of the mpi* executables in my env confused our scheduler, but the simple deletion of them from bin/ after installing xESMF also worked for me.

And, of course, avoid ever putting the lib/ directory in any sort of LD_LIBRARY_PATH and putting the include/ directory in any sort of INCLUDE path. I can only imagine the support horror if an mpi.mod or libmpi.a from Python got into another build!