Macros file is incorrect when you switch from MPI to mpi-serial MPILIB -- have to delete by hand

[ekluzek]

The Macros file is slightly different on yellowstone with intel when you create a case with MPI and then change to mpi-serial. You then have to delete the old one by hand and create a new one when you do this, ./case.setup -clean won't delete it for you. Maybe there should be an option to case.setup to delete the Macros file to make the need to do this more obvious?

I think the reason for this is in case you added customization's to your Macros file so it doesn't delete it. But, you could also have the case.setup -clean save a copy. Most users won't be customizing their Macros file.

Here are sample differences between the two, the use of MPICC and MPIFC for SCC and SFC cause the build of MCT to fail.

[erik@yslogin5 ERP_D_P15x2_Ld3.f10_f10.ICRUCLM50BGC.yellowstone_intel.clm-luna.GC.160603-155752]$ diff Macros ../ERS_Ld5.1x1_vancouverCAN.I1PTCLM50.yellowstone_intel.clm-default.GC.160603-155752/
8c8
< CPPDEFS+= -DFORTRANUNDERSCORE -DNO_R16 -DINTEL_MKL -DHAVE_SSE2  -DLINUX -DCPRINTEL 

---
> CPPDEFS+= -DFORTRANUNDERSCORE -DNO_R16 -DLINUX -DCPRINTEL 
10,12c10
< SLIBS+= -Wl,-rpath ${PAPI_LIB} -L${PAPI_LIB} -lpapi
< 
< CFLAGS:=  -no-opt-dynamic-align -fp-model precise -std=gnu99   -xHost 

---
> CFLAGS:=  -no-opt-dynamic-align -fp-model precise -std=gnu99 
20c18
< FFLAGS:= -no-opt-dynamic-align  -convert big_endian -assume byterecl -ftz -traceback -assume realloc_lhs -fp-model source    -xHost 

---
> FFLAGS:= -no-opt-dynamic-align  -convert big_endian -assume byterecl -ftz -traceback -assume realloc_lhs -fp-model source  
34c32
< MPICXX:=mpiicpc

---
> MPICXX:= mpicxx 
40,43d37
< PAPI_INC:= /glade/apps/opt/papi/5.3.0/intel/12.1.5/include/
< 
< PAPI_LIB:=/glade/apps/opt/papi/5.3.0/intel/12.1.5/lib64 
< 
50c44
< SCC:=$(MPICC)

---
> SCC:= icc 
54c48
< SFC:=$(MPIFC)

---
> SFC:= ifort 
58,59d51
< TRILINOS_PATH:=$(TRILINOS_PATH)
< 

Another way to handle this would be to modify the construction of the Macros file so that there aren't required differences between mpi-serial and non mpi-serial.

(note from @mnlevy1981: I edited this to change the formatting)

[ekluzek]

In the CSEG meeting, we decided that Sean would work on this in the context of making sure that the Macros file doesn't have to change when MPILIB gets changed. There are some other conditions that shouldn't change the Macros file either.

[quantheory]

I don't have my yubikey access fixed yet, so I don't have Yellowstone access, but I believe that this commit should fix the issue:

https://github.com/quantheory/cime/commit/753163a10a3c87cd2ea1342f42827a669d7663fa

Note that this commit should change answers for serial cases due to compiler-optimization changes on yellowstone. In effect, we're not optimizing serial builds as heavily as MPI builds right now, and this commit would bring the two back into alignment. The PEA test is supposed to catch this sort of discrepancy, but currently it builds the mpi-serial executable from the same case as the parallel executable, it does not wipe the Macros file between one build and the next. We could tweak the PEA test to do this.

I'm working on a change that causes the Macros file to be finished at create_newcase time, so that it never has to be removed and regenerated due to env_build settings (even if you change the compiler). That would solve this problem in a more permanent way.

[billsacks]

Closing this old issue; I'm not sure if it has been fixed. Feel free to reopen in ESMCI/cime if necessary.