search

easybuilders / easybuild-easyblocks

Collection of easyblocks that implement support for building and installing software with EasyBuild.
https://easybuild.io
GNU General Public License v2.0
105 stars 285 forks source link

CPK2 with toolchains with MKL pick up MKL FFT not the dependency FFTW #2332

Open hajgato opened 3 years ago

hajgato commented 3 years ago

this is the link command:

mpiifort  -D__parallel -D__BLACS -D__SCALAPACK -D__FFTSG  -D__LIBXSMM  -D__LIBINT -D__LIBXC -D__INTEL -D__FFTW3 -D__PLUMED2 -I/apps/gent/RHEL8/zen2-ib/software/imkl/2020.0.166-iimpi-2020a/mkl/include -I/apps/gent/RHEL8/zen2-ib/software/FFTW/3.3.8-intel-2020a/include   -static-intel  -nofor-main   -L/tmp/vsc40003/easybuild/CP2K/7.1/intel-2020a/cp2k-7.1/lib/Linux-x86-64-intel/popt -o /tmp/vsc40003/easybuild/CP2K/7.1/intel-2020a/cp2k-7.1/exe/Linux-x86-64-intel/cp2k.popt cp2k.o  -lcp2kstart -lcp2kmc -lcp2kswarm -lcp2kmotion -lcp2kthermostat -lcp2kemd -lcp2ktmc -lcp2kmain -lcp2kgrid -ldbcsrarnoldi -ldbcsrx -lcp2kshg_int -lcp2keri_mme -lcp2kminimax -lcp2khfxbase -lcp2ksubsys -lcp2kxc -lcp2kao -lcp2kpw_env -lcp2kinput -lcp2kpw -lcp2kfft -lcp2kfpga -lcp2kfm -lcp2kcommon -lcp2kmpiwrap -lcp2kbase -L/tmp/vsc40003/easybuild/CP2K/7.1/intel-2020a/cp2k-7.1/lib/Linux-x86-64-intel/popt/exts/dbcsr -ldbcsr -Wl,--start-group -liomp5 -lpthread /apps/gent/RHEL8/zen2-ib/software/Libint/2.6.0-iimpi-2020a-lmax-6-cp2k/lib/libint2.a -lstdc++  -L/apps/gent/RHEL8/zen2-ib/software/libxc/4.3.4-iccifort-2020.1.217/lib -lxcf03 -lxc -lxsmm -lxsmmf -Wl,-Bstatic  -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core  -Wl,-Bdynamic -L/apps/gent/RHEL8/zen2-ib/software/FFTW/3.3.8-intel-2020a/lib -lfftw3 -lplumed -Wl,--end-group

However, FFTW is after MKL, and it should be before:

... -L/apps/gent/RHEL8/zen2-ib/software/FFTW/3.3.8-intel-2020a/lib -lfftw3  -Wl,-Bstatic  -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core  -Wl,-Bdynamic ...

the same is true for includes: -I/apps/gent/RHEL8/zen2-ib/software/FFTW/3.3.8-intel-2020a/include should be before -I/apps/gent/RHEL8/zen2-ib/software/imkl/2020.0.166-iimpi-2020a/mkl/include (or the MKL FFT include should be not there.)

boegel commented 3 years ago

Hmm, I'm not sure the order in which libraries are linked really determine which FFT implementation is being used by CP2K...

In both cases, the cp2k.popt binary will link to both libfftw3.so.3 and statically linking to MKL.

CP2K is compiled with -D__FFTW3 when FFTW is included as a dependency (both with or without MKL), and that determines which functions are being used I think...

@hajgato How did you come to the conclusion that CP2K isn't actually leveraging FFTW3 when MKL is included in the toolchain?

hajgato commented 3 years ago

@boegel Just checked the order of linked libraries

boegel commented 3 years ago

Just to complicate matters even more, apparently CP2K provides runtime control over which FFT library is used, see https://manual.cp2k.org/cp2k-2_1-branch/CP2K_INPUT/GLOBAL.html#desc_PREFERRED_FFT_LIBRARY ... (hat tip @smoors)