[FYI] building BAGEL on Cray machines

[jeffhammond]

NERSC asked me for help with this, so I thought I'd share it in case anyone else ever wants to know it. I did not attempt to figure out how to add it to the documentation properly.

BAGEL installation

> ../configure MPICC=cc CC=cc MPICXX=CC CXX=CC \
CXXFLAGS="-std=c++17" --with-mpi=intel --enable-mkl \
--with-boost=/usr/common/software/boost/1.67.0/intel/haswell \
CPPFLAGS="-I$MKLROOT/include" && make -j12

Note that BAGEL's build system doesn't care that I am using Cray MPI rather than Intel MPI, because the MPI libraries and headers are provided by the cc and CC wrappers, which behave like mpicc and mpicxx on non-Cray systems.

Dependencies

I installed the Intel 19.4 compiler in my directory, so that is what is used, not Intel 18 as shown below.

I unload the cray-libsci module because it conflicts with MKL. I had to unload GCC 7.3.0 here because of a conflict with Boost, but I'm not sure if that really matters to BAGEL.

> module list
Currently Loaded Modulefiles:
  1) modules/3.2.10.6                                 10) gni-headers/5.0.12.0-6.0.7.1_3.11__g3b1768f.ari  19) cray-mpich/7.7.3
  2) nsg/1.2.0                                        11) xpmem/2.2.15-6.0.7.1_5.11__g7549d06.ari          20) craype-hugepages2M
  3) intel/18.0.1.163                                 12) job/2.2.3-6.0.7.1_5.44__g6c4e934.ari             21) craype-accel-host
  4) craype-network-aries                             13) dvs/2.7_2.2.120-6.0.7.1_12.1__g74cb2cc4          22) git/2.15.1
  5) craype/2.5.15                                    14) alps/6.6.43-6.0.7.1_5.46__ga796da32.ari          23) pgi/18.10
  6) udreg/2.3.2-6.0.7.1_5.13__g5196236.ari           15) rca/2.2.18-6.0.7.1_5.48__g2aa4f39.ari            24) fftw/2.1.5.9
  7) ugni/6.0.14.0-6.0.7.1_3.13__gea11d3d.ari         16) atp/2.1.3                                        25) cray-memkind
  8) pmi/5.0.14                                       17) PrgEnv-intel/6.0.4                               26) altd/2.0
  9) dmapp/7.1.1-6.0.7.1_6.2__g45d1b37.ari            18) craype-haswell                                   

Additional details

BAGEL is relatively large and NERSC has rather limited filesystem quotas, so I had to clone the repo into my project directory, because my home directory was not able to hold the source and build files. This may be a consequence of having multiple NWChem folders already 😉

[shiozaki]

Thanks Jeff, I will leave this open so others can see it. This is somewhat system dependent, so I am hoping NERSC is going to have a page about installing BAGEL one day :-)

Note to add: The only reason why MPI has to be specified as a configure option (--with-mpi=xxx) is to make sure we link proper MKL libraries (one can see it in configure.ac). I presume most of the MPI implementation other than OpenMPI can be handled as if they are the same as IntelMPI.