MPAS-Dev / MPAS-Model

Repository for MPAS models and shared framework releases.
235 stars 312 forks source link

Enable the use of the more modern 'mpi_f08' module for MPI #1142

Closed mgduda closed 6 months ago

mgduda commented 6 months ago

This PR enables MPAS to make use of the more modern mpi_f08 module introduced in MPI-3.0.

The top-level Makefile now tries to compile a test program that uses mpi_f08, and if that test is successful, the macro MPAS_USE_MPI_F08 is defined in the CPPFLAGS used in the build; otherwise, MPAS will make use of the mpi module as it previously did.

With the use of the mpi_f08 module, certain MPI types are no longer integers in Fortran, but are derived types; e.g., MPI_Comm, MPI_Request, MPI_Datatype, and MPI_Info. However, in some instances, an integer-typed MPI type is still needed for interoperability, and the MPI standard permits this to be done through the mpi_val member of all MPI derived types, e.g., MPI_Comm % mpi_val.

mgduda commented 6 months ago

The motivation for the changes in this PR arises from compilation errors with newer GNU compilers when using the mpi module from the cray-mpich library that is available on NCAR's Derecho system:

mpas_dmpar.F:5115:25:

 5115 |           call MPI_Irecv(commListPtr % ibuffer, commListPtr % nList, MPI_INTEGERKIND, commListPtr % procID, commListPtr % procID, dminfo % comm, commListPtr % reqID, mpi_ierr)
      |                         1
......
 7892 |             call MPI_Irecv(commListPtr % rbuffer, commListPtr % nList, MPI_REALKIND, commListPtr % procID, commListPtr % procID, &
      |                           2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(4)/REAL(4)).

While these specific errors can be avoided by adding the -fallow-argument-mismatch flag for the gfortran compiler, it was deemed sub-optimal to simply demote errors to warnings.

Since there are combinations of compilers and MPI implementations on Derecho that still don't provide an mpi_f08 module (e.g., nvhpc/24.1 and cray-mpich/8.1.27), we still require support for the older mpi module, justifying the use of preprocessing directives to activate the use of either mpi or mpi_f08.