PrincetonUniversity / athena-public-version

(MOVED) Athena++ GRMHD code and adaptive mesh refinement (AMR) framework. Active repository --->
https://github.com/PrincetonUniversity/athena
BSD 3-Clause "New" or "Revised" License
160 stars 117 forks source link

Configuring with -mpi overrides --ccmd=mpiicpc #37

Closed trwaters closed 4 years ago

trwaters commented 4 years ago

On LANL's Grizzly, I have loaded 1) intel/19.0.4 2) intel-mpi/2019.4

On the master branch, doing python configure.py --prob=linear_wave --coord=cartesian --cxx=icc --ccmd=mpiicpc gives Compilation command: mpiicpc -O3 -std=c++11 -ipo -xhost -inline-forceinline -qopenmp-simd -qopt-prefetch=4 -qoverride-limits -diag-disable 3180

But of course I need the -mpi flag, and upon adding it, i.e. with python configure.py --prob=linear_wave --coord=cartesian --cxx=icc --ccmd=mpiicpc -mpi I get instead Compilation command: mpicxx -O3 -std=c++11 -ipo -xhost -inline-forceinline -qopenmp-simd -qopt-prefetch=4 -qoverride-limits -diag-disable 3180

That is, mpicxx instead of mpiicpc . This defaulting to the GNU compiler with -mpi does not happen in athena-public-version-1.1.1

felker commented 4 years ago

Try switching the order of the flags?

trwaters commented 4 years ago

Same behavior. Can you reproduce?

msbc commented 4 years ago

You can always manually edit the Makefile as a workaround.

trwaters commented 4 years ago

Right, manually editing the Makefile is what led me to uncover this issue.

c-white commented 4 years ago

Try --mpiccmd instead of --ccmd. I forget exactly why these are differentiated, but I believe there was some reason for it.

trwaters commented 4 years ago

Thanks Chris, that resolves the issue.

felker commented 4 years ago

Oh yes, I misread the report.

I added --mpiccmd in https://github.com/PrincetonUniversity/athena/pull/178 so that both --ccmd=/usr/local/bin/gcc could be passed to the hybrid_linwave.py regression test and not affect the MPI wrapper compiler builds.

trwaters commented 4 years ago

Okay. FYI, I see this is mentioned in the docs for configuring, but there are two occurrences of these lines and only the 2nd mentions "when not using -mpi" for --ccmd

--cxx [compiler] : select a C++ compiler and predefined compiler flags --ccmd [compiler command] : set a compiler command overriding --cxx

felker commented 4 years ago

Thanks, fixed this in the Wiki