search

E3SM-Project / E3SM

Energy Exascale Earth System Model source code. NOTE: use "maint" branches for your work. Head of master is not validated.
https://docs.e3sm.org/E3SM
Other
343 stars 352 forks source link

case.setup error with mvapich2 on Compy #3922

Open dqwu opened 3 years ago

dqwu commented 3 years ago

For Intel compiler

ls /share/apps/mvapich2/2.3.1/intel
18.0.0  19.0.3  19.0.4  20.0.0

...
./create_newcase --compiler=intel --mpilib=mvapich2 --case intel_mvapich2_X_f19_g16 --compset X --res f19_g16
cd intel_mvapich2_X_f19_g16
./case.setup

ERROR: module command /share/apps/modules/bin/modulecmd python load cmake/3.11.4 intel/19.0.5 mvapich2/2.3.1 netcdf/4.6.3 pnetcdf/1.9.0 mkl/2019u5 failed with message:
The combination of compiler and MPI does not appear to be built yet

For PGI compiler

ls /share/apps/mvapich2/2.3.1/pgi
18.10  19.1  19.4c  19.5  19.7

...
./create_newcase --compiler=pgi --mpilib=mvapich2 --case pgi_mvapich2_X_f19_g16 --compset X --res f19_g16
cd pgi_mvapich2_X_f19_g16
./case.setup

ERROR: module command /share/apps/modules/bin/modulecmd python load cmake/3.11.4 pgi/19.10 mvapich2/2.3.1 netcdf/4.6.3 pnetcdf/1.9.0 mkl/2019u5 failed with message:
The combination of compiler and MPI does not appear to be built yet

So for mvapich2/2.3.1, it cannot combine with intel/19.0.5 or pgi/19.10 so far.

Related PRs: Makes Intel 19.0.5 as a default compiler on Compy (PR #3210) Updates PGI compiler to version 19.10 (PR #3382)

singhbalwinder commented 3 years ago

Hi @dqwu : We generally ask for only intel MPI builds with the new compilers as @amametjanov did some experiments to demonstrate that intelMPI works better than mvapich2 on Compy (intel architecture). I can ask support to build mvapich2 library and associated pnetcdf libraries using intel19.0.5/pgi19.10, if you need to use mvapich2.

dqwu commented 3 years ago

Hi @dqwu : We generally ask for only intel MPI builds with the new compilers as @amametjanov did some experiments to demonstrate that intelMPI works better than mvapich2 on Compy (intel architecture). I can ask support to build mvapich2 library and associated pnetcdf libraries using intel19.0.5/pgi19.10, if you need to use mvapich2.

Thanks for the information. I think another workaround is to modify the machine file to let mvapich2 on Compy use intel/19.04 and pgi/19.7 (intelMPI can still use intel/19.0.5 and pgi/19.10).

singhbalwinder commented 3 years ago

PGI/19.7 may be okay but Intel/19.0.4 and 19.0.3 are buggy compilers. They unexpectedly change the answers in the CLUBB model. That is why we moved to 19.0.5.