Errors compiling on gpu cluster

[pablo-unzueta]

Hello!

I'm trying to compile qe-gipaw on a gpu cluster. After following the instructions for compilation, I get the following errors:

ftn -acc -cuda -gpu=cc80,cuda11.0 -acc -o gipaw.x gipaw_module.o gipaw_main.o paw_gipaw.o stop_code.o gipaw_setup.o gipaw_routines.o gipaw_routines_bands.o greenfunction.o orthoatwfc1.o cgsolve_all.o h_psiq.o cg_psi.o symmetrize_rho.o symmetrize_field.o ch_psi_all.o compute_u_kq.o suscept_crystal.o j_para.o biot_savart.o init_gipaw_1.o init_gipaw_2.o init_us_2_no_phase.o write_tensor_field.o velocity.o nmr_routines.o epr_routines.o efg.o hyperfine.o core_relax.o util.o atomic.o knight_shift.o mossbauer.o gipaw_version.o output_magres.o gipaw_results.o xml_routines.o /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/PW/src/libpw.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/Modules/libqemod.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/MBD/libmbd.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/Davidson/libdavid.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/CG/libcg.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/PPCG/libppcg.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/libks_solvers.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/FFTXlib/libqefft.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/LAXlib/libqela.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/UtilXlib/libutil.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/upflib/libupf.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/dft-d3/libdftd3qe.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/LR_Modules/liblrmod.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/XClib/xc_lib.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/UtilXlib/libutil.a -L/global/common/software/nersc/pm-2021q4/sw/libxc/v5.2.2/pm-gpu/lib -lxcf03 -lxc -cudalib=cufft,cublas,cusolver,curand /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0//external/devxlib/src/libdevXlib.a /global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0//EIGENSOLVER_GPU/lib_eigsolve/lib_eigsolve.a -L/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0//external/devxlib/src -ldevXlib -L/opt/cray/libfabric/1.11.0.4.114/lib64 -lopenblas -L/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0//FoX/lib -lFoX_dom -lFoX_sax -lFoX_wxml -lFoX_common -lFoX_utils -lFoX_fsys -lopenblas
nvlink error : Multiple definition of 'gpu_threaded_backassign_380_gpu' in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/libks_solvers.a:generic_cublas.o', first defined in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/PPCG/libppcg.a:generic_cublas.o' nvlink error : Multiple definition of 'gpu_threaded_backassign_371_gpu' in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/libks_solvers.a:generic_cublas.o', first defined in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/PPCG/libppcg.a:generic_cublas.o' nvlink error : Multiple definition of 'gpu_threaded_assign_327_gpu' in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/libks_solvers.a:generic_cublas.o', first defined in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/PPCG/libppcg.a:generic_cublas.o' nvlink error : Multiple definition of 'gpu_threaded_assign_320_gpu' in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/libks_solvers.a:generic_cublas.o', first defined in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/PPCG/libppcg.a:generic_cublas.o' nvlink error : Multiple definition of 'gpu_threaded_memset_272_gpu' in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/libks_solvers.a:generic_cublas.o', first defined in '/global/common/software/nersc/pm-2021q4/sw/qe/qe-7.0/KS_Solvers/PPCG/libppcg.a:generic_cublas.o' nvlink fatal : merge_elf failed pgacclnk: child process exit status 2: /opt/nvidia/hpc_sdk/Linux_x86_64/21.11/compilers/bin/tools/nvdd make[1]: [Makefile:62: gipaw.x] Error 2 make[1]: Leaving directory '/global/u2/p/punzu001/software/qe-gipaw/src' make: [Makefile:7: build] Error 2

This seems to be related to this post on the nvidia forums: https://forums.developer.nvidia.com/t/nvlink-error-multiple-definition-errors-when-linking-to-the-same-library-twice/62892

Any help to remedy this compilation would be greatly appreciated!

[dceresoli]

Hi. At this stage QE-GIPAW does not support GPUs. The porting to GPU is in progress, hopefully we'll get a postdoc working on it. D.