Closed saipavan27 closed 2 years ago
If you want to avoid using an MPI wrapper, try QMP with comms_type=SINGLE and chroma arch=parscalar.
I have built QMP with the flag you specified,
../configure --prefix=/opt/commstype_single/qmp --with-qmp-comms-type=SINGLE CC=gcc CFLAGS="-O3 -march=core2"
and used this while building QUDA, but getting
`In file included from /home/quda/lib/interface_quda.cpp:29:0:
/home/quda/include/mpi_comm_handle.h:5:10: fatal error: mpi.h: No such file or directory
compilation terminated.` this error. Ideally it should not use MPI because we didn't specify comms_type=MPI right? Please suggest on what to be done.
I think QUDA can be built with either MPI or QMP support (and it might default to MPI). QMP sounds right. If that doesn't work installing an MPI implementation and building everything in arch=parscalar should work.
So to clarify, what you are saying is we cannot run Chroma without QMP or MPI right?
I believe QUDA is the more restrictive component here which requires either of them.
But, I have run QUDA test cases without QMP/MPI successfully. So, I suppose it is not issue with QUDA.
I am building Chroma with QUDA,QDP and without QMP,MPI
Build steps followed: QUDA: (QMP=OFF) git clone --recursive https://github.com/lattice/quda.git mkdir build_quda cd build_quda
CXX=g++ CC=gcc cmake ../quda -DQUDA_GPU_ARCH=sm_70 -DQUDA_DIRAC_STAGGERED=ON -DQUDA_DIRAC_DOMAIN_WALL=OFF -DQUDA_DIRAC_TWISTED_MASS=OFF -DQUDA_DOWNLOAD_EIGEN=ON -DQUDA_DIRAC_WILSON=ON -DQUDA_DIRAC_CLOVER=ON -DQUDA_DIRAC_TWISTED_CLOVER=ON -DQUDA_DYNAMIC_CLOVER=ON -DQUDA_DIRAC_CLOVER_HASENBUSCH=ON -DQUDA_LINK_HISQ=OFF -DQUDA_MULTIGRID=ON -DQUDA_MPI=OFF -DQUDA_INTERFACE_MILC=OFF -DQUDA_QMP=OFF -DQUDA_QIO=OFF -DQUDA_BUILD_SHAREDLIB=ON -DQUDA_BUILD_ALL_TESTS=ON -DQUDA_TEX=OFF -DCMAKE_BUILD_TYPE=DEVEL -DCMAKE_CXX_FLAGS="-O3 -march=core2 -funroll-all-loops -fargument-noalias-global -finline-limit=50000 -fpeel-loops" -DCMAKE_CFLAGS="-O3 -fargument-noalias-global -funroll-all-loops -fpeel-loops -march=core2"
make -jQDP: (QMP=OFF) git clone --recursive https://github.com/usqcd-software/qdpxx.git cd qdpxx autoreconf -i mkdir build cd build
../configure --prefix=/opt/singlegpu/qdpxx --enable-parallel-arch=scalar --enable-precision=single --enable-sse2 CXX=g++ CC=gcc CXXFLAGS="-O3 -march=core2 -funroll-all-loops -fargument-noalias-global -finline-limit=50000 -fpeel-loops" CFLAGS="-O3 -fargument-noalias-global -funroll-all-loops -fpeel-loops -march=core2"
make -j make installChroma: git clone --recursive https://github.com/JeffersonLab/chroma.git -b devel cd chroma autoreconf -i mkdir build cd build
../configure --prefix=/opt/singlegpu/chroma --with-qdp=/opt/singlegpu/qdpxx --with-quda=/home/build_quda --enable-gmp --enable-sse2 --enable-sse3 CC=gcc CXX=g++ CXXFLAGS="-O3 -march=core2 -funroll-all-loops -fargument-noalias-global -finline-limit=50000 -fpeel-loops" CFLAGS="-O3 -fargument-noalias-global -funroll-all-loops -fpeel-loops -march=core2"
make -j make install
Build got successful.
Error: When I run any test case, getting the below error Invalid device number -1 (/home/quda/lib/interface_quda.cpp:559 in initQudaDevice()) last kernel called was (name=,volume=,aux=)
When I build with QMP ON, its working.It is picking the device ID correctly. So, can we use Chroma without QMP and if so, please suggest what I am missing.
System Configuration CUDA version: 10.1 gcc version: 7.5 cmake version: 3.16.5 OS: Ubuntu 16.04.1
Thanks in advance :)