Does Chroma support without QMP/MPI on single GPU?

[saipavan27]

I am building Chroma with QUDA,QDP and without QMP,MPI

Build steps followed: QUDA: (QMP=OFF) git clone --recursive https://github.com/lattice/quda.git mkdir build_quda cd build_quda CXX=g++ CC=gcc cmake ../quda -DQUDA_GPU_ARCH=sm_70 -DQUDA_DIRAC_STAGGERED=ON -DQUDA_DIRAC_DOMAIN_WALL=OFF -DQUDA_DIRAC_TWISTED_MASS=OFF -DQUDA_DOWNLOAD_EIGEN=ON -DQUDA_DIRAC_WILSON=ON -DQUDA_DIRAC_CLOVER=ON -DQUDA_DIRAC_TWISTED_CLOVER=ON -DQUDA_DYNAMIC_CLOVER=ON -DQUDA_DIRAC_CLOVER_HASENBUSCH=ON -DQUDA_LINK_HISQ=OFF -DQUDA_MULTIGRID=ON -DQUDA_MPI=OFF -DQUDA_INTERFACE_MILC=OFF -DQUDA_QMP=OFF -DQUDA_QIO=OFF -DQUDA_BUILD_SHAREDLIB=ON -DQUDA_BUILD_ALL_TESTS=ON -DQUDA_TEX=OFF -DCMAKE_BUILD_TYPE=DEVEL -DCMAKE_CXX_FLAGS="-O3 -march=core2 -funroll-all-loops -fargument-noalias-global -finline-limit=50000 -fpeel-loops" -DCMAKE_CFLAGS="-O3 -fargument-noalias-global -funroll-all-loops -fpeel-loops -march=core2" make -j

QDP: (QMP=OFF) git clone --recursive https://github.com/usqcd-software/qdpxx.git cd qdpxx autoreconf -i mkdir build cd build ../configure --prefix=/opt/singlegpu/qdpxx --enable-parallel-arch=scalar --enable-precision=single --enable-sse2 CXX=g++ CC=gcc CXXFLAGS="-O3 -march=core2 -funroll-all-loops -fargument-noalias-global -finline-limit=50000 -fpeel-loops" CFLAGS="-O3 -fargument-noalias-global -funroll-all-loops -fpeel-loops -march=core2" make -j make install

Chroma: git clone --recursive https://github.com/JeffersonLab/chroma.git -b devel cd chroma autoreconf -i mkdir build cd build ../configure --prefix=/opt/singlegpu/chroma --with-qdp=/opt/singlegpu/qdpxx --with-quda=/home/build_quda --enable-gmp --enable-sse2 --enable-sse3 CC=gcc CXX=g++ CXXFLAGS="-O3 -march=core2 -funroll-all-loops -fargument-noalias-global -finline-limit=50000 -fpeel-loops" CFLAGS="-O3 -fargument-noalias-global -funroll-all-loops -fpeel-loops -march=core2"

make -j make install

Build got successful.

Error: When I run any test case, getting the below error Invalid device number -1 (/home/quda/lib/interface_quda.cpp:559 in initQudaDevice()) last kernel called was (name=,volume=,aux=)

When I build with QMP ON, its working.It is picking the device ID correctly. So, can we use Chroma without QMP and if so, please suggest what I am missing.

System Configuration CUDA version: 10.1 gcc version: 7.5 cmake version: 3.16.5 OS: Ubuntu 16.04.1

Thanks in advance :)

[fwinter]

If you want to avoid using an MPI wrapper, try QMP with comms_type=SINGLE and chroma arch=parscalar.

[saipavan27]

I have built QMP with the flag you specified, ../configure --prefix=/opt/commstype_single/qmp --with-qmp-comms-type=SINGLE CC=gcc CFLAGS="-O3 -march=core2" and used this while building QUDA, but getting `In file included from /home/quda/lib/interface_quda.cpp:29:0: /home/quda/include/mpi_comm_handle.h:5:10: fatal error: mpi.h: No such file or directory

include

compilation terminated.` this error. Ideally it should not use MPI because we didn't specify comms_type=MPI right? Please suggest on what to be done.

[fwinter]

I think QUDA can be built with either MPI or QMP support (and it might default to MPI). QMP sounds right. If that doesn't work installing an MPI implementation and building everything in arch=parscalar should work.

[saipavan27]

So to clarify, what you are saying is we cannot run Chroma without QMP or MPI right?

[fwinter]

I believe QUDA is the more restrictive component here which requires either of them.

[saipavan27]

But, I have run QUDA test cases without QMP/MPI successfully. So, I suppose it is not issue with QUDA.