OP-DSL / OP2-Common

OP2: open-source framework for the execution of unstructured grid applications on clusters of GPUs or multi-core CPUs
https://op-dsl.github.io
Other
98 stars 46 forks source link

The library can not be found when I compile OP2 #227

Open lj-cug opened 2 years ago

lj-cug commented 2 years ago

Dear Sir: The code structure of OP2, especially the compiling bash, has been reorganized. After I set all the environment variables include compiler, third-party libraries, then run "make detect" in op2. The problem happened like this: I'm sure I have set ENV. variables correctedly. Another environment variables should I config? I use Ubuntu 18.04 OS. Compilers: . C: gcc . C++: g++ . CUDA: /usr/local/cuda/bin/nvcc . Fortran: gfortran

MPI compilers: . C: /mnt/lijian/3rd-library-install/mpich-3.3/bin/mpicc . C++: /mnt/lijian/3rd-library-install/mpich-3.3/bin/mpicxx . Fortran: /mnt/lijian/3rd-library-install/mpich-3.3/bin/mpif90

CUDA libraries: not found

HDF5 I/O: . Sequential: not found . Parallel: not found

MPI partitioners: . PT-Scotch: not found . ParMETIS: not found

Compilation flags: . C: -std=c99 -MMD -MP -Wall -Wextra -pedantic -O3 -DOMPI_SKIP_MPICXX -DMPICH_IGNORE_CXX_SEEK -DMPIPP_H . C++: -MMD -MP -Wall -Wextra -pedantic -O3 -DOMPI_SKIP_MPICXX -DMPICH_IGNORE_CXX_SEEK -DMPIPP_H . CUDA: -gencode arch=compute_60,code=sm_60 -m64 -Xptxas=-v -O3 -use_fast_math . Fortran: -Wall -pedantic -ffixed-line-length-none -ffree-line-length-none -fcray-pointer

bozbez commented 2 years ago

Could you post the list of environment variables you are setting? They will need to point to directories with the include and lib folders of the dependencies with the lib folder containing the correct compiled libraries (e.g. libmetis.a and libparmetis.a for ParMETIS).

The build logic will try to build and link test executables with the given flags, which could also fail if the compiled libraries were produced using an incompatible compiler. You can check why the compile test is failing by removing the redirection of stderr to /dev/null in the X_TEST variable definition in the makefiles/dependencies/*.mk makefiles.

lj-cug commented 2 years ago

Dear Buddy, all the libraries are compiled using GNU compiler. If I config C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, the libraries can be linked in right way. It's weird. And I also noticed some variables like HAVE_CUDA, CUDA_LIB... My laptop is installed with Ubuntu OS. What's your OS?

My environment variables configuration is following:

options for compiling OP2 library

export OP2_C_COMPILER=gnu export OP2_C_CUDA_COMPILER=nvhpc export OP2_F_COMPILER=gnu

export OP2_INSTALL_PATH=/home/lijian/DSL/OP2-Common/op2/ unset OP_AUTO_SOA

External libraries

export PT_SCOTCH_INSTALL_PATH=/home/lijian/DSL/ptscotch export PARMETIS_INSTALL_PATH=/home/lijian/DSL/parmetis-4.0 export LD_LIBRARY_PATH=$PARMETIS_INSTALL_PATH/lib:$PT_SCOTCH_INSTALL_PATH/lib:$LD_LIBRARY_PATH export HDF5_PAR_INSTALL_PATH=/usr/lib/x86_64-linux-gnu/hdf5/mpich export LD_LIBRARY_PATH=$HDF5_PAR_INSTALL_PATH/lib:$LD_LIBRARY_PATH

NVIDIA CUDA

export PATH=/usr/local/cuda/bin:$PATH export CUDA_INSTALL_PATH=/usr/local/cuda export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH export NV_ARCH=Pascal # Geforce MX150

set mpich-3.3 or openmpi-3.1

export MPI_INSTALL_PATH=$MY_APPs/mpich-3.3 export LD_LIBRARY_PATH=$MPI_INSTALL_PATH/lib:$LD_LIBRARY_PATH export PATH=$MY_APPs/mpich-3.3/bin:$PATH

bozbez commented 2 years ago

Strange, would expect that to work. For PT-Scotch it should be PTSCOTCH_INSTALL_PATH not PT_SCOTCH_... - it was wrong in the docs but should be fixed now. I've added a make variable to help debug the detection: could you update your repo and then attach the full terminal output of: make MAKE_DETECT_DEBUG=1 SHELL='bash -x' detect?

lj-cug commented 2 years ago

I correct PTSCOTCH_INSTALL_PATH and update my git repo. Then I run make MAKE_DETECT_DEBUG=1 SHELL='bash -x' detect The problems are : /usr/bin/ld: can not find -lmpicxx -lparmetis -lscoth .... The environment variables are set as above, there's no problem. Now I have to set the environment variables of include and lib paths of the third-party libraries manually as following, the OP2 can be compiled in normal way. I don't know the bash code to compile OP2 in this repo. In last year, I compile OP2 in the older repo.

include path

export C_INCLUDE_PATH=$CUDA_INSTALL_PATH/include:$MPI_INSTALL_PATH/include:$HDF5_PAR_INSTALL_PATH/include:$PTSCOTCH_INSTALL_PATH/include:$CPLUS_INCLUDE_PATH export CPLUS_INCLUDE_PATH=$CUDA_INSTALL_PATH/include:$MPI_INSTALL_PATH/include:$HDF5_PAR_INSTALL_PATH/include:$PTSCOTCH_INSTALL_PATH/include:$CPLUS_INCLUDE_PATH

library path

export HAVE_CUDA=true export CUDA_LIB=/usr/local/cuda/lib64 export HAVE_PTSCOTCH=true export PTSCOTCH_LIB=$PTSCOTCH_INSTALL_PATH/lib export HAVE_PARMETIS=true export PARMETIS_LIB=$PARMETIS_INSTALL_PATH/lib export HAVE_HDF5_PAR=true export HDF5_PAR_LIB=$HDF5_PAR_INSTALL_PATH/lib