Failed to compile wannier_tools with self compiled libarpack_LINUX.a.

[hongyi-zhao]

On Ubuntu 20.04, I try to compile the git master version of wannier_tools with intel mkl and intel MPI as shown below:

1. Compile arpack library:

$ mkdir -p ~/www.caam.rice.edu/software
$ cd ~/www.caam.rice.edu/software
$ curl -x socks5://127.0.0.1:18888 -O https://www.caam.rice.edu/software/ARPACK/SRC/arpack96.tar.gz
$ curl -x socks5://127.0.0.1:18888 -O https://www.caam.rice.edu/software/ARPACK/SRC/patch.tar.gz

$ tar zxvf arpack96.tar.gz
$ tar zxvf patch.tar.gz

Then adjust ARmake.inc as below according to the notes here for working with ifort and mkl:

###########################################################################
#
#  Program:         ARPACK
#
#  Module:          ARmake.inc
#
#  Purpose:         Top-level Definitions
#
#  Creation date:   February 22, 1996
#
#  Modified:
#
#  Send bug reports, comments or suggestions to arpack@caam.rice.edu
#
############################################################################
#
# %---------------------------------%
# |  SECTION 1: PATHS AND LIBRARIES |
# %---------------------------------%
#
#
# %--------------------------------------%
# | You should change the definition of  |
# | home if ARPACK is built some place   | 
# | other than your home directory.      |
# %--------------------------------------%
#
#home = $(HOME)/ARPACK

home = $(HOME)/www.caam.rice.edu/software/ARPACK

#
#  %--------------------------------------%
#  | The platform identifier to suffix to |
#  | the end of library names             |
#  %--------------------------------------%
#
#PLAT = SUN4
PLAT = LINUX

#
#  %------------------------------------------------------%
#  | The directories to find the various pieces of ARPACK |
#  %------------------------------------------------------%
#
BLASdir      = $(home)/BLAS
LAPACKdir    = $(home)/LAPACK
UTILdir      = $(home)/UTIL
SRCdir       = $(home)/SRC
#
DIRS        = $(BLASdir) $(LAPACKdir) $(UTILdir) $(SRCdir)
#
# %-------------------------------------------------------------------%
# | Comment out the previous line and uncomment the following         |
# | if you already have the BLAS and LAPACK installed on your system. |
# | NOTE: ARPACK assumes the use of LAPACK version 2 codes.           |
# %-------------------------------------------------------------------%
#
#DIRS         = $(UTILdir) $(SRCdir)
#
# %---------------------------------------------------%
# | The name of the libraries to be created/linked to |
# %---------------------------------------------------%
#
ARPACKLIB  = $(home)/libarpack_$(PLAT).a
LAPACKLIB = 
BLASLIB = 
#
ALIBS =  $(ARPACKLIB) $(LAPACKLIB) $(BLASLIB) 
#
# 
# %---------------------------------------------------------%
# |                  SECTION 2: COMPILERS                   |
# |                                                         |
# | The following macros specify compilers, linker/loaders, |
# | the archiver, and their options.  You need to make sure |
# | these are correct for your system.                      |
# %---------------------------------------------------------%
#
#
# %------------------------------%
# | Make our own suffixes' list. |
# %------------------------------%
#
.SUFFIXES:
.SUFFIXES:  .f  .o
#
# %------------------%
# | Default command. |
# %------------------%
#
.DEFAULT:
    @$(ECHO) "Unknown target $@, try:  make help"
#
# %-------------------------------------------%
# |  Command to build .o files from .f files. |
# %-------------------------------------------%
#
.f.o:
    @$(ECHO) Making $@ from $<
    @$(FC) -c $(FFLAGS) $<
#
# %-----------------------------------------%
# | Various compilation programs and flags. |
# | You need to make sure these are correct |
# | for your system.                        |
# %-----------------------------------------%
#
#FC      = f77
FC       = ifort
#FFLAGS = -O -cg89
FFLAGS  = -O3

LDFLAGS = 
CD      = cd

ECHO    = echo

LN      = ln
LNFLAGS = -s

MAKE    = /bin/make

RM      = rm
RMFLAGS = -f

SHELL   = /bin/sh
#
#  %----------------------------------------------------------------%
#  | The archiver and the flag(s) to use when building an archive   |
#  | (library).  Also the ranlib routine.  If your system has no    |
#  | ranlib, set RANLIB = touch.                                    |
#  %----------------------------------------------------------------%
#
AR = ar 
ARFLAGS = rv
#RANLIB  = touch
RANLIB   = ranlib
#
# %----------------------------------%
# | This is the general help target. |
# %----------------------------------%
#
help:
    @$(ECHO) "usage: make ?"

Finally, compile the library as below:

$ module load intel/oneapi
$ make all

2. Compile wannier_tools:

$ git clone https://github.com/quanshengwu/wannier_tools.git wannier_tools.git 
$ cd wannier_tools.git/src

Replace the ARPACK=/Users/quan/work/workplace/ARPACK/libarpack_MAC.a line in Makefile.intel-mpi with the following one:

ARPACK=/home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a

Then compile wannier_tools:

$ module load intel/oneapi
$ make -f Makefile.intel-mpi 
mpiifort -fpp -DMPI -DINTELMKL -fpe3 -O3 -static-intel module.o sparse.o wt_aux.o math_lib.o symmetry.o readHmnR.o inverse.o proteus.o eigen.o ham_qlayer2qlayer.o psi.o unfolding.o rand.o ham_slab.o ham_bulk.o ek_slab.o ek_bulk_polar.o ek_bulk.o readinput.o fermisurface.o surfgreen.o surfstat.o mat_mul.o ham_ribbon.o ek_ribbon.o fermiarc.o berrycurvature.o wanniercenter.o dos.o  orbital_momenta.o landau_level_sparse.o landau_level.o lanczos_sparse.o berry.o wanniercenter_adaptive.o effective_mass.o findnodes.o sigma_OHE.o sigma.o sigma_AHC.o main.o -o wt.x -Wl,--start-group /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_sequential.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl /home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a
ld: /home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a(znaupd.o): in function `znaupd_':
znaupd.f:(.text+0x269): undefined reference to `dlamch_'
ld: /home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a(zneupd.o): in function `zneupd_':
zneupd.f:(.text+0x6c): undefined reference to `dlamch_'
ld: zneupd.f:(.text+0xa2e): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0xa75): undefined reference to `zlaset_'
ld: zneupd.f:(.text+0xaf0): undefined reference to `zlahqr_'
ld: zneupd.f:(.text+0xb32): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0xc88): undefined reference to `ztrsen_'
ld: zneupd.f:(.text+0xd2a): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0xda8): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0xded): undefined reference to `zgeqr2_'
ld: zneupd.f:(.text+0xe53): undefined reference to `zunm2r_'
ld: zneupd.f:(.text+0xe8c): undefined reference to `zlacpy_'
ld: zneupd.f:(.text+0xf21): undefined reference to `zscal_'
ld: zneupd.f:(.text+0xf5d): undefined reference to `zscal_'
ld: zneupd.f:(.text+0x11b3): undefined reference to `ztrevc_'
ld: zneupd.f:(.text+0x128e): undefined reference to `zdscal_'
ld: zneupd.f:(.text+0x134e): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0x13d1): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0x1451): undefined reference to `ztrmm_'
ld: zneupd.f:(.text+0x1491): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0x14b9): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0x14f8): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0x2fe1): undefined reference to `zgeru_'
ld: zneupd.f:(.text+0x30a7): undefined reference to `zscal_'
ld: zneupd.f:(.text+0x3100): undefined reference to `zscal_'
ld: /home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a(znaup2.o): in function `znaup2_':
znaup2.f:(.text+0x9a): undefined reference to `dlamch_'
ld: znaup2.f:(.text+0x371): undefined reference to `zcopy_'
ld: znaup2.f:(.text+0x3aa): undefined reference to `zcopy_'
ld: znaup2.f:(.text+0x7e1): undefined reference to `zcopy_'
ld: znaup2.f:(.text+0x917): undefined reference to `zcopy_'
ld: znaup2.f:(.text+0x101c): undefined reference to `zcopy_'
ld: /home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a(zneigh.o): in function `zneigh_':
zneigh.f:(.text+0x95): undefined reference to `zlacpy_'
ld: zneigh.f:(.text+0xc3): undefined reference to `zlaset_'
ld: zneigh.f:(.text+0x123): undefined reference to `zlahqr_'
ld: zneigh.f:(.text+0x15a): undefined reference to `zcopy_'
ld: zneigh.f:(.text+0x217): undefined reference to `ztrevc_'
ld: zneigh.f:(.text+0x293): undefined reference to `zdscal_'
ld: zneigh.f:(.text+0x2e3): undefined reference to `zcopy_'
ld: zneigh.f:(.text+0x333): undefined reference to `zcopy_'
ld: zneigh.f:(.text+0x34a): undefined reference to `zdscal_'
ld: /home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a(zgetv0.o): in function `zgetv0_':
zgetv0.f:(.text+0xb0): undefined reference to `zlarnv_'
ld: zgetv0.f:(.text+0x15f): undefined reference to `zcopy_'
ld: zgetv0.f:(.text+0x1b7): undefined reference to `zcopy_'
ld: zgetv0.f:(.text+0x4a2): undefined reference to `zgemv_'
ld: zgetv0.f:(.text+0x4f7): undefined reference to `zgemv_'
ld: zgetv0.f:(.text+0x534): undefined reference to `zcopy_'
ld: zgetv0.f:(.text+0x57f): undefined reference to `zcopy_'
ld: zgetv0.f:(.text+0x680): undefined reference to `zcopy_'
ld: /home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a(znaitr.o): in function `znaitr_':
znaitr.f:(.text+0x141): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x1fb): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x389): undefined reference to `zgemv_'
ld: znaitr.f:(.text+0x3e2): undefined reference to `zgemv_'
ld: znaitr.f:(.text+0x47f): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x4dc): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x70b): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x75b): undefined reference to `zdscal_'
ld: znaitr.f:(.text+0x78a): undefined reference to `zdscal_'
ld: znaitr.f:(.text+0x7f6): undefined reference to `zlascl_'
ld: znaitr.f:(.text+0x859): undefined reference to `zlascl_'
ld: znaitr.f:(.text+0x8c0): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0xd5a): undefined reference to `zgemv_'
ld: znaitr.f:(.text+0xd9d): undefined reference to `zgemv_'
ld: znaitr.f:(.text+0xe23): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x10ed): undefined reference to `zlanhs_'
ld: znaitr.f:(.text+0x114f): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x120f): undefined reference to `dlamch_'
ld: znaitr.f:(.text+0x123f): undefined reference to `dlabad_'
ld: znaitr.f:(.text+0x1250): undefined reference to `dlamch_'
ld: /home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a(znapps.o): in function `znapps_':
znapps.f:(.text+0xb9): undefined reference to `zlaset_'
ld: znapps.f:(.text+0x3ec): undefined reference to `zlartg_'
ld: znapps.f:(.text+0x8bc): undefined reference to `zscal_'
ld: znapps.f:(.text+0x8f8): undefined reference to `zscal_'
ld: znapps.f:(.text+0x932): undefined reference to `zscal_'
ld: znapps.f:(.text+0xae7): undefined reference to `zgemv_'
ld: znapps.f:(.text+0xb9b): undefined reference to `zgemv_'
ld: znapps.f:(.text+0xbd4): undefined reference to `zcopy_'
ld: znapps.f:(.text+0xc53): undefined reference to `zlacpy_'
ld: znapps.f:(.text+0xcb0): undefined reference to `zcopy_'
ld: znapps.f:(.text+0xcf6): undefined reference to `zscal_'
ld: znapps.f:(.text+0xec3): undefined reference to `zlanhs_'
ld: znapps.f:(.text+0x101a): undefined reference to `zlanhs_'
ld: znapps.f:(.text+0x104e): undefined reference to `dlamch_'
ld: znapps.f:(.text+0x107e): undefined reference to `dlabad_'
ld: znapps.f:(.text+0x108f): undefined reference to `dlamch_'
make: *** [Makefile.intel-mpi:37: main] Error 1

Any hints for this problem will be highly appreciated.

Regards, HY

[hongyi-zhao]

I also tried to use arpack-ng, see here for details of the discussion.

[quanshengwu]

@hongyi-zhao Could you try to run nm -a /home/werner/www.caam.rice.edu/software/ARPACK/libarpack_LINUX.a |grep zcopy and put the output here?

[hongyi-zhao]

This problem has been solved in the above-mentioned issue filed on arpack-ng, see here for details of the discussion.

To be specific, the makefile used by wannier_tools needs to adopt the following change, i.e., putting libarpack.a and/or libparpack.a before the intel MKL library group when using static libraries:

LIBS =${ARPACK} \
      -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a \
      ${MKLROOT}/lib/intel64/libmkl_sequential.a \
      ${MKLROOT}/lib/intel64/libmkl_core.a \
      -Wl,--end-group -lpthread -lm -ldl

Anyway, I post here the info you request:

$ nm -a libarpack_LINUX.a | grep zcopy
                 U zcopy_
                 U zcopy_
                 U zcopy_
                 U zcopy_
                 U zcopy_
                 U zcopy_
zcopy.o:
0000000000000000 T zcopy_
0000000000000000 a zcopy.f
                 U zcopy_
                 U zcopy_
                 U zcopy_

BTW, I still failed to compile the libparpack.a for the original ARPACK version obtained from here, as shown below:

$ module load intel/oneapi
$ make parpacklib 
( cd ; /bin/make single double complex complex16 )
make[1]: Entering directory '/home/werner'
make[1]: *** No rule to make target 'single'.  Stop.
make[1]: Leaving directory '/home/werner'
make: *** [Makefile:69: parpacklib] Error 2

[quanshengwu]

@hongyi-zhao Thank you very much for your suggestion. I am glad that your issue was solved.

By the way, we don't PARPACK in WannierTools so far. So you don't have to compile the parpacklib.

[hongyi-zhao]

By the way, we don't PARPACK in WannierTools so far. So you don't have to compile the parpacklib.

I just tried this out of curiosity.

BTW, I think that It is best to make the appropriate corrections/notes in the Makefile templates supplied in this repo.