search

TRIQS / triqs_0.x

DEPRECATED -- This is the repository of the older versions of TRIQS
Other
11 stars 9 forks source link

MKL linking #9

Closed aeantipov closed 12 years ago

aeantipov commented 12 years ago

Hi, everyone. I still try to get TRIQS installed to a machine with MKL. The mpi compiler is GNU mpic++, g++ and gcc are used as CXX and CC. From the FindBLAS.Cmake I got that TRIQS actually looks for an "mkl" string in LAPACKLIBS and if it exists, replaces cblas with mkl_cblas etc. After some playing I got string "You are using mkl" during configuration and finally everything went smooth and I got no errors or warnings during generating CMakeCache. Here are the lines, that I have put in ccmake (CMakeCache.txt is attached):

LAPACK_64_BIT                    OFF
MKL_INCLUDE_DIR                  /usr/local/intel/composerxe/2011.7.256/mkl/include
LAPACK_LIBRARY                   /usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64
LAPACK_LIBS                      /usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_core.so;/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_intel_ilp64.so; /usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_sequential.so;/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so

CmakeCache - http://gist.github.com/1363901

The problem occurs during linking CXX executable a_x_ty_nopy. The log is in the bottom. To me it looks like there are some dependencies on pthreads and openmp (libgomp) for GNU case. There were some more mistakes with linking but they have passed after I included more libraries in LAPACK_LIBS. So this suggests that some additional flags should be passed when linking. I tried to pass "-fopenmp -lpthread" in CMAKE_MODULE_LINKER_FLAGS but it didn't help.

Thanks for the support!

Build log:

[ 39%] Built target boost_for_triqs
Linking CXX shared module mpi.so
[ 45%] Built target mpi
[ 48%] Built target triqs
Linking CXX executable a_x_ty_nopy
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_destroy_hash'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `omp_get_nested'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_provide_receive_buffer'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_hash_compare_ptrs'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_sections_start'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_num_send_tokens'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_parallel_start'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_ordered_start'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_loop_ordered_dynamic_next'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_bcopy'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_get_node_id'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_atomic_end'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_single_start'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_set_acceptable_sizes'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_critical_start'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_critical_name_end'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_open'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_crc'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_hash_hash_ptr'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_blocking_receive_no_spin'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_create_hash'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_close'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_deregister_memory'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_loop_end'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_loop_ordered_static_next'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_barrier'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_dma_free'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_drop_sends'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_atomic_start'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_loop_dynamic_start'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `omp_get_num_procs'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_directed_send_with_callback'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_num_receive_tokens'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_critical_end'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_hash_remove'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_dma_malloc'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_node_id_to_host_name'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `omp_get_max_threads'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_unknown'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_create_lookaside'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_get_host_name'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_allow_remote_memory_access'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_strerror'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_loop_ordered_dynamic_start'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_hash_insert'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `_gm_assertion_failed'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_blocking_receive'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_bzero'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `omp_set_nested'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_sections_end_nowait'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_ordered_end'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `omp_in_parallel'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_critical_name_start'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_loop_end_nowait'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_loop_ordered_static_start'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_lookaside_free'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_perror'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_provide_receive_buffer_with_tag'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_hash_find'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `omp_get_thread_num'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_destroy_lookaside'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_loop_dynamic_next'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_sections_next'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_parallel_end'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_lookaside_alloc'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `GM_PAGE_LEN'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_receive'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `omp_get_num_threads'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_node_id_to_unique_id'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_send_with_callback'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_register_memory'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_host_name_to_node_id'
../../../../foreignlibs/boost/libboost_for_triqs.so: undefined reference to `gm_unique_id_to_node_id'
/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64/libmkl_gnu_thread.so: undefined reference to `GOMP_parallel_sections_start'
collect2: ld returned 1 exit status
make[2]: *** [triqs/arrays/test/C++/a_x_ty_nopy] Error 1
make[1]: *** [triqs/arrays/test/C++/CMakeFiles/a_x_ty_nopy.dir/all] Error 2
make: *** [all] Error 2
mferrero commented 12 years ago

Try to do the following and let me know what you get (I turned off any open mp lib and did not link against mkl_gnu_thread which I think is not necessary):

cmake -DBLAS_LIBRARY=-lmkl_core -DLAPACK_LIBRARY="-lmkl_intel_lp64;-lmkl_sequential;-lpthread" -DCBLAS_INCLUDE_DIR=/usr/local/intel/composerxe/2011.7.256/mkl/include -DLAPACK_LINKER_FLAGS=-L/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64 -DBOOST_SOURCE_DIR=path_to_boost_sources path_to_triqs_sources

aeantipov commented 12 years ago

This time it's much better. At first I got bunch of errors, which told something like 'undefined reference to `gm_crc'. Adding -lgm to LAPACK_LIBRARY fixed it and the compilation got to the building of Wien2k module, at which it failed. I used this short shell script for configuration:

CXX="g++" CC="gcc" cmake \
-DBLAS_LIBRARY=-lmkl_core -DLAPACK_LIBRARY="-lgm;-lmkl_intel_lp64;-lmkl_sequential;-lpthread" \
-DCBLAS_INCLUDE_DIR=/usr/local/intel/composerxe/2011.7.256/mkl/include \
-DLAPACK_LINKER_FLAGS=-L/usr/local/intel/composerxe/2011.7.256/mkl/lib/intel64 \
-DBOOST_SOURCE_DIR=/home/antipov/code/boost_svn \
-DMPIEXEC:FILEPATH=/usr/local/mpich/gnu/bin/mpiexec \
-DMPI_COMPILER:FILEPATH=/usr/local/mpich/gnu/bin/mpic++ \
-DMPI_LIBRARY:FILEPATH=/usr/local/mpich/gnu/lib/libmpich.a \
-DMPI_INCLUDE_PATH:STRING=/usr/local/mpich/gnu/include \
-DCMAKE_INSTALL_PREFIX=/home/antipov/_local \
/home/antipov/code/triqs

The result is:

[ 39%] Built target boost_for_triqs
[ 45%] Built target mpi
Linking CXX shared library libtriqs.so
[ 48%] Built target triqs
Linking CXX executable a_x_ty_nopy
[ 48%] Built target a_x_ty_nopy
Scanning dependencies of target array_cache_nopy
[ 48%] Building CXX object triqs/arrays/test/C++/CMakeFiles/array_cache_nopy.dir/array_cache.cpp.o
Linking CXX executable array_cache_nopy
[ 48%] Built target array_cache_nopy
........
Linking Fortran executable dmftproj
[ 95%] Built target dmftproj
Scanning dependencies of target triqs_Wien2k
[ 95%] Generating vertex.so
See /home/antipov/Downloads/triqs_build3/pytriqs/Wien2k/F90/f2pyBuild.log for logs
make[2]: *** [pytriqs/Wien2k/F90/vertex.so] Error 1
make[1]: *** [pytriqs/Wien2k/F90/CMakeFiles/triqs_Wien2k.dir/all] Error 2
make: *** [all] Error 2

f2pyBuild.log:

running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "vertex" sources
creating /tmp/tmpcFezih
creating /tmp/tmpcFezih/src.linux-x86_64-2.6
f2py options: []
f2py: vertex.pyf
Reading fortran codes...
    Reading file 'vertex.pyf' (format:free)
Post-processing...
    Block: vertex
            Block: u4ind
Post-processing (stage 2)...
Building modules...
    Building module "vertex"...
        Constructing wrapper function "u4ind"...
          u_out = u4ind(rcl,tm,[l,n])
    Wrote C/API module "vertex" to file "/tmp/tmpcFezih/src.linux-x86_64-2.6/vertexmodule.c"
  adding '/tmp/tmpcFezih/src.linux-x86_64-2.6/fortranobject.c' to sources.
  adding '/tmp/tmpcFezih/src.linux-x86_64-2.6' to include_dirs.
copying /usr/lib64/python2.6/site-packages/numpy/f2py/src/fortranobject.c -> /tmp/tmpcFezih/src.linux-x86_64-2.6
copying /usr/lib64/python2.6/site-packages/numpy/f2py/src/fortranobject.h -> /tmp/tmpcFezih/src.linux-x86_64-2.6
build_src: building npy-pkg config files
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize GnuFCompiler
Could not locate executable g77
Could not locate executable f77
customize IntelFCompiler
Found executable /usr/bin/gfortran
Found executable /usr/local/bin/ifort
customize LaheyFCompiler
Could not locate executable lf95
lahey: no Fortran 77 compiler found
lahey: no Fortran 77 compiler found
customize PGroupFCompiler
Found executable /usr/local/pgi/linux86-64/11.9/bin/pgf77
Found executable /usr/local/pgi/linux86-64/11.9/bin/pgf90
customize PGroupFCompiler
customize PGroupFCompiler using build_ext
building 'vertex' extension
compiling C sources
C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -fPIC

creating /tmp/tmpcFezih/tmp
creating /tmp/tmpcFezih/tmp/tmpcFezih
creating /tmp/tmpcFezih/tmp/tmpcFezih/src.linux-x86_64-2.6
compile options: '-I/tmp/tmpcFezih/src.linux-x86_64-2.6 -I/usr/lib64/python2.6/site-packages/numpy/core/include -I/usr/include/python2.6 -c'
gcc: /tmp/tmpcFezih/src.linux-x86_64-2.6/fortranobject.c
gcc: /tmp/tmpcFezih/src.linux-x86_64-2.6/vertexmodule.c
compiling Fortran sources
Fortran f77 compiler: /usr/local/pgi/linux86-64/11.9/bin/pgf77 -fpic -Minform=inform -Mnosecond_underscore -fast
Fortran f90 compiler: /usr/bin/gfortran -fpic -Minform=inform -Mnosecond_underscore -fast
Fortran fix compiler: /usr/bin/gfortran -Mfixed -fpic -Minform=inform -Mnosecond_underscore -fast
compile options: '-I/tmp/tmpcFezih/src.linux-x86_64-2.6 -I/usr/lib64/python2.6/site-packages/numpy/core/include -I/usr/include/python2.6 -c'
gfortran:f90: vertex.f90
Warning: Using -M  is deprecated, use -J instead
Warning: Using -M  is deprecated, use -J instead
f951: error: unrecognized command line option "-fast"
Fatal Error: gfortran: Only one -J option allowed
Warning: Using -M  is deprecated, use -J instead
Warning: Using -M  is deprecated, use -J instead
f951: error: unrecognized command line option "-fast"
Fatal Error: gfortran: Only one -J option allowed
error: Command "/usr/bin/gfortran -fpic -Minform=inform -Mnosecond_underscore -fast -I/tmp/tmpcFezih/src.linux-x86_64-2.6 -I/usr/lib64/python2.6/site-packages/numpy/core/include -I/usr/include/python2.6 -c -c vertex.f90 -o /tmp/tmpcFezih/vertex.o" failed with exit status 1

gfortran has version 4.5

aichhorn commented 12 years ago

There might be an issue with the fortran compiler used in f2py. You use gnu compilers, but mkl libraries. I got similar errors with plain f2py, when specifiying only the fortran exectuable to be 'ifort', for instance, and not the vendor to be 'intel'. That resulted in strange linking behavior and errors. I don't know, how to fix this easily in cmake, though.

aeantipov commented 12 years ago

Thanks, Markus. That is right - somehow instead of using gfortran whilst building f2py the configuration script found pgi compiler in my path and invoked it as a f77 compiler, which was wrong. As soon as I changed my PATH variable to avoid it, everything finally compiled smoothly. Once again, thanks everyone for helping with this!