gms-bbg / gamess-issues

GAMESS issue tracking
7 stars 1 forks source link

Problem with execution of libcchem (gamess.cchem.01.x) using openmpi #46

Open acandac opened 3 years ago

acandac commented 3 years ago

Hi!

I have installed LIBCCHEM using openmpi-4.0.3. Installation and Linking gamess goes smoothly and generated /usr/local/gamess/gamess.cchem.01.x file. I also set TARGET=ga in rungms-dev file. When I try to run games on a GTX1080ti GPU using the following command : rungms-dev test 01 1 1 > test.log &

I get the following error message :

orterun --mca btl vader,self,tcp -np 1 --bind-to none --map-by ppr:1:node --report-bindings /usr/local/gamess/gamess.cchem.01.x [cadd0atjenk:04680] MCW rank 0 is not bound (or bound to all available processors) /usr/local/gamess/gamess.cchem.01.x[0x2be2bbd] /usr/local/gamess/gamess.cchem.01.x[0x2be3c23] /lib64/libc.so.6(+0x361a0)[0x7fea6d4dc1a0] /usr/local/gamess/gamess.cchem.01.x[0x224ec88] /usr/local/gamess/gamess.cchem.01.x[0x22b5bf1] /usr/local/gamess/gamess.cchem.01.x[0x1d1662b] /usr/local/gamess/gamess.cchem.01.x[0x1d1904b] /usr/lib64/libgomp.so.1(+0x1783e)[0x7fea7567883e] /lib64/libpthread.so.0(+0x7569)[0x7fea7544a569] /lib64/libc.so.6(clone+0x3f)[0x7fea6d59e85f] MPI_ABORT was invoked on rank 0 in communicator MPI COMMUNICATOR 4 DUP FROM 0 with errorcode 8. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

I think rungms-dev is kind of looking for a CPU but not a GPU and it is unable to link to a CPU , would that be the case?

Before running rungms-dev, I set CCHEM env as follows.

export CCHEM="devices=0;memory=6g"

So my GPU is already defined in env.

Well, I got stuck here. Any help will be greatly appreciated.

Best, Jenk

My system configurations are OS: OpenSUse LeaP 15.0 ; compiler : gfortran 7.4 ; math library : intelMKL

İnput file : ! restrained optimization and mp2 postprocessing $contrl scftyp=rhf runtyp=optimize mplevl=2 ispher=1 coord=zmt nzvar=0 icut=11 $end $system mwords=400 memddi=7000 $end $statpt opttol=0.00001 IACTAT(1)=19,-32 $end $basis gbasis=acct $end $guess guess=huckel $end $SCF DIRSCF=.true. CONV=2.5D-7 FDIFF=.FALSE. $end $TRANS DIRTRF=.T. $END $data m3u C1 H
O 1 0.9600000 .... .... .... $end

P.S. the input file above works with CPU sockets installation of gamess.00.x by setting memddi=0

Install.info : setenv GMS_PATH /usr/local/gamess setenv GMS_BUILD_DIR /usr/local/gamess setenv GMS_TARGET linux64 setenv GMS_HPC_SYSTEM_TARGET generic setenv GMS_FORTRAN gfortran setenv GMS_GFORTRAN_VERNO 7.4 setenv GMS_MATHLIB mkl setenv GMS_MATHLIB_PATH /opt/intel/compilers_and_libraries_2019.5.281/linux/mkl/lib/intel64 setenv GMS_MKL_VERNO 12 setenv GMS_DDI_COMM mpi setenv GMS_MPI_LIB openmpi setenv GMS_MPI_PATH /usr/local/openmpi-4.0.3 setenv GMS_MSUCC false setenv GMS_LIBCCHEM true setenv GMS_LIBCCHEM_GPU_SUPPORT true setenv GMS_CCHEM_HF true setenv GMS_CCHEM_MP2 true setenv GMS_CCHEM_RI true setenv GMS_CCHEM_CC true setenv GMS_HDF5_PATH /usr/local/gamess-cuda-lib/hdf5 setenv GMS_CUDA_BOARD pascal setenv GMS_CUDA_PATH /usr/local/cuda-10.0 setenv GMS_LIBCCHEM_LIBINT false setenv GMS_LIBCCHEM_LIBS -ldl setenv GMS_EIGEN_PATH /usr/local/gamess-cuda-lib/eigen-3.3.7 setenv GMS_BOOST_PATH /usr/local/gamess/libcchem/externals/boost setenv GMS_GA_PATH /usr/local/gamess/libcchem/externals/ga setenv GMS_PHI none setenv GMS_SHMTYPE sysv setenv GMS_OPENMP false setenv GMS_LIBXC true setenv GMS_FPE_FLAGS '-ffpe-trap=invalid,zero,overflow'