search

awslabs / palace

3D finite element solver for computational electromagnetics
https://awslabs.github.io/palace/dev
Apache License 2.0
245 stars 50 forks source link

libCEED build error #100

Closed EMinsight closed 1 year ago

EMinsight commented 1 year ago

There is a build issue for libCEED that is missing the compiler flag -fPIC.

After adding the flag in ExternalLibCEED.cmake file, the whole building passes.

Platform: ubuntu 22.04

sebastiangrimberg commented 1 year ago

Hi @EMinsight, thanks for reporting this.

The Makefile in libCEED should be adding this flag as needed (see here), so I'm surprised you're seeing this issue. I can confirm in my setups that the flag is there.

Is there anything else non-standard about your setup? What compilers are you using? Is this a source build using Spack or are you running CMake and performing the build from source yourself?

EMinsight commented 1 year ago

Hi @sebastiangrimberg , I found the root issue.

The makefile in libCEED adds the -fPIC flag if STATIC=0(see here), but the CMake pass "STATIC=" to make, there is no value at the right of the equal sign. It should be like this(STATIC=0).

BRs, James

sebastiangrimberg commented 1 year ago

What version of make are you using? If I define STATIC=0, the check in the libCEED Makefile does not add -fPIC and builds the static library, which is not what we want. With STATIC=, the check is correct (the dynamic library is built). This is true on my local machine (macOS) but also on Ubuntu as tested in the CI builds.

EMinsight commented 1 year ago

make version is 4.3 CMake version is 3.27.3

sebastiangrimberg commented 1 year ago

Thanks. The libCEED build should also be dynamic and not static no matter what value of BUILD_SHARED_LIBS is given. Can you confirm you're seeing issues for both cases? Also, does the Spack build work or is that failing too on your machine? What's the compiler you are using?

Sorry for all the questions, but I'm unable to reproduce the issue on your same setup so am somewhat suspicious. Building with STATIC= is indeed the correct syntax for dynamic library builds, not STATIC=0. The build should work fine by the info you've given me.

sebastiangrimberg commented 1 year ago

In order to further debugging, it would be helpful if you could provide the full output of running cmake in your build directory (from scratch), and also the output of the build (make). Using this I can hopefully discern what's going on. Thanks!

EMinsight commented 1 year ago

This is error message

/usr/bin/ld: build/interface/ceed-basis.o: relocation R_X86_64_PC32 against symbol `CeedEvalModes' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status gmake[3]: [Makefile:552: build/libceed.so] Error 1 make[2]: [CMakeFiles/libCEED.dir/build.make:105: extern/libCEED-cmake/src/libCEED-stamp/libCEED-install] Error 2 make[1]: [CMakeFiles/Makefile2:152: CMakeFiles/libCEED.dir/all] Error 2 make: [Makefile:91: all] Error 2

I uploaded the whole build directory and you can get all things. Please get the package here

BTW, how could I build the master from Spack? The compiler is GCC@11.4.0

sebastiangrimberg commented 1 year ago

I don't need the entire build directory, can you just send me the terminal log? I'm not sure if that info can be extracted out of the build directory.

For spack, you can spack install palace@develop to pull the tip of main.

Also, regarding the STATIC=0 solution, this is in general not correct. See the docs: https://www.gnu.org/software/make/manual/html_node/Conditional-Functions.html. In particular:

The first argument, condition, first has all preceding and trailing whitespace stripped, then is expanded. If it expands to any non-empty string, then the condition is considered to be true. If it expands to an empty string, the condition is considered to be false.

So indeed STATIC= is what we want.

EMinsight commented 1 year ago

CMake output:

-- The CXX compiler identification is GNU 11.4.0
-- The C compiler identification is GNU 11.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Setting CMAKE_INSTALL_PREFIX to '/home/tcad/workspace/code/palace/build' as none was specified
-- Setting CMAKE_BUILD_TYPE to 'Release' as none was specified
-- Setting BUILD_SHARED_LIBS to 'OFF' as it was not specified
-- ====================== Configuring MPI dependency ======================
-- Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1") 
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version "3.1") 
-- Found MPI: TRUE (found version "3.1")  
-- ================= Configuring BLAS/LAPACK dependencies =================
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: /usr/lib/x86_64-linux-gnu/libblas.so  
-- Looking for cheev_
-- Looking for cheev_ - not found
-- Looking for cheev_
-- Looking for cheev_ - found
-- Found LAPACK: /usr/lib/x86_64-linux-gnu/liblapack.so;/usr/lib/x86_64-linux-gnu/libblas.so  
-- Using BLAS/LAPACK located by CMake
-- ================ Configuring METIS/ParMETIS dependencies ===============
-- GKLIB_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_C_COMPILER=/usr/bin/cc; -DCMAKE_C_FLAGS=
-- METIS_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_C_COMPILER=/usr/bin/cc; -DCMAKE_C_FLAGS=; -DGKlib_ROOT=/home/tcad/workspace/code/palace/build
-- PARMETIS_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_C_COMPILER=/usr/bin/cc; -DCMAKE_C_FLAGS=; -DGKlib_ROOT=/home/tcad/workspace/code/palace/build; -Dmetis_ROOT=/home/tcad/workspace/code/palace/build
-- ================= Configuring SuperLU_DIST dependency ==================
-- SUPERLU_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_CXX_COMPILER=/usr/bin/c++; -DCMAKE_CXX_FLAGS=; -DCMAKE_C_COMPILER=/usr/bin/cc; -DCMAKE_C_FLAGS=; -DXSDK_ENABLE_Fortran=OFF; -Denable_tests=OFF; -Denable_examples=OFF; -Denable_double=ON; -Denable_single=ON; -Denable_complex16=ON; -Denable_openmp=OFF; -DTPL_ENABLE_PARMETISLIB=ON; -DTPL_PARMETIS_LIBRARIES=/home/tcad/workspace/code/palace/build/lib/libparmetis.a$<SEMICOLON>/home/tcad/workspace/code/palace/build/lib/libmetis.a$<SEMICOLON>/home/tcad/workspace/code/palace/build/lib/libGKlib.a; -DTPL_PARMETIS_INCLUDE_DIRS=/home/tcad/workspace/code/palace/build/include; -DTPL_ENABLE_COMBBLASLIB=OFF; -DTPL_ENABLE_CUDALIB=OFF; -DTPL_ENABLE_HIPLIB=OFF; -DTPL_ENABLE_LAPACKLIB=ON; -DTPL_ENABLE_INTERNAL_BLASLIB=OFF; -DLAPACK_LIBRARIES=/usr/lib/x86_64-linux-gnu/liblapack.so$<SEMICOLON>/usr/lib/x86_64-linux-gnu/libblas.so; -DBLAS_LIBRARIES=/usr/lib/x86_64-linux-gnu/liblapack.so$<SEMICOLON>/usr/lib/x86_64-linux-gnu/libblas.so
-- =============== Configuring PETSc and SLEPc dependencies ===============
-- PETSC_OPTIONS: COPTFLAGS=; CXXOPTFLAGS=; --prefix=/home/tcad/workspace/code/palace/build; --with-cc=/usr/bin/cc; --with-cxx=/usr/bin/c++; --with-fc=0; --with-scalar-type=complex; --with-precision=double; --with-clanguage=c; --with-x=0; --with-debugging=0; --with-shared-libraries=0; --known-64-bit-blas-indices=0; --with-mpi-lib=[/usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so,/usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so]; --with-mpi-include=[/usr/lib/x86_64-linux-gnu/openmpi/include,/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi]; --with-blaslapack-lib=[/usr/lib/x86_64-linux-gnu/liblapack.so,/usr/lib/x86_64-linux-gnu/libblas.so]; --with-blaslapack-include=[/usr/include/x86_64-linux-gnu]
-- SLEPC_OPTIONS: --prefix=/home/tcad/workspace/code/palace/build; --with-feast=0; --with-arpack=0
-- ==================== Configuring LIBXSMM dependency ====================
-- LIBXSMM_OPTIONS: PREFIX=/home/tcad/workspace/code/palace/build; CC=/usr/bin/cc; CXX=/usr/bin/c++; FC=0; FORTRAN=0; BLAS=0; SYM=1; VERBOSE=1; PPKGDIR=lib/pkgconfig; PMODDIR=lib/pkgconfig; STATIC=0
-- ===================== Configuring HYPRE dependency =====================
-- HYPRE_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_C_COMPILER=/usr/bin/cc; -DCMAKE_C_FLAGS=; -DHYPRE_ENABLE_SHARED=OFF; -DHYPRE_ENABLE_MIXEDINT=OFF; -DHYPRE_ENABLE_BIGINT=OFF; -DHYPRE_WITH_MPI=ON; -DHYPRE_WITH_OPENMP=OFF; -DHYPRE_ENABLE_HYPRE_BLAS=OFF; -DHYPRE_ENABLE_HYPRE_LAPACK=OFF; -DLAPACK_LIBRARIES=/usr/lib/x86_64-linux-gnu/liblapack.so$<SEMICOLON>/usr/lib/x86_64-linux-gnu/libblas.so; -DBLAS_LIBRARIES=/usr/lib/x86_64-linux-gnu/liblapack.so$<SEMICOLON>/usr/lib/x86_64-linux-gnu/libblas.so
-- ================= Configuring nlohmann/json dependency =================
-- JSON_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_CXX_COMPILER=/usr/bin/c++; -DCMAKE_CXX_FLAGS=; -DJSON_Install=ON; -DJSON_BuildTests=OFF
-- ====================== Configuring fmt dependency ======================
-- FMT_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_CXX_COMPILER=/usr/bin/c++; -DCMAKE_CXX_FLAGS=; -DFMT_INSTALL=ON; -DFMT_DOC=OFF; -DFMT_TEST=OFF
-- ===================== Configuring Eigen dependency =====================
-- EIGEN_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_CXX_COMPILER=/usr/bin/c++; -DCMAKE_CXX_FLAGS=; -DEIGEN_BUILD_DOC=ON; -DBUILD_TESTING=OFF
-- ===================== Configuring GSLIB dependency =====================
-- GSLIB_OPTIONS: INSTALL_ROOT=/home/tcad/workspace/code/palace/build; CC=/usr/bin/cc; MPI=1; STATIC=1; SHARED=0; BLAS=0; CFLAGS= -I/usr/lib/x86_64-linux-gnu/openmpi/include -I/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi; LDFLAGS= /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so
-- ==================== Configuring libCEED dependency ====================
-- Performing Test SUPPORTS_OMP_SIMD
-- Performing Test SUPPORTS_OMP_SIMD - Success
-- LIBCEED_OPTIONS: prefix=/home/tcad/workspace/code/palace/build; CC=/usr/bin/cc; OPT= -fopenmp-simd; STATIC=; VERBOSE=1; CUDA_DIR=/disable-cuda; XSMM_DIR=/home/tcad/workspace/code/palace/build; BLAS_LIB=
-- ====================== Configuring MFEM dependency =====================
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")  
-- Building MFEM with zlib support for binary output compression
-- MFEM_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_CXX_COMPILER=/usr/bin/c++; -DCMAKE_CXX_FLAGS=; -DMFEM_USE_MPI=YES; -DMFEM_USE_OPENMP=OFF; -DMFEM_THREAD_SAFE=OFF; -DMFEM_USE_SUPERLU=ON; -DMFEM_USE_STRUMPACK=OFF; -DMFEM_USE_MUMPS=OFF; -DMFEM_USE_ZLIB=YES; -DMFEM_USE_LIBUNWIND=NO; -DMFEM_USE_METIS_5=YES; -DBLAS_LIBRARIES=/usr/lib/x86_64-linux-gnu/liblapack.so$<SEMICOLON>/usr/lib/x86_64-linux-gnu/libblas.so; -DLAPACK_LIBRARIES=/usr/lib/x86_64-linux-gnu/liblapack.so$<SEMICOLON>/usr/lib/x86_64-linux-gnu/libblas.so; -DMFEM_USE_CEED=YES; -DCEED_DIR=/home/tcad/workspace/code/palace/build; -DlibCEED_REQUIRED_LIBRARIES=/home/tcad/workspace/code/palace/build/lib/libxsmm.so; -DMFEM_USE_GSLIB=YES; -DGSLIB_DIR=/home/tcad/workspace/code/palace/build; -DMETIS_LIBRARIES=/home/tcad/workspace/code/palace/build/lib/libmetis.a$<SEMICOLON>/home/tcad/workspace/code/palace/build/lib/libGKlib.a; -DMETIS_INCLUDE_DIRS=/home/tcad/workspace/code/palace/build/include; -DHYPRE_DIR=/home/tcad/workspace/code/palace/build; -DHYPRE_REQUIRED_PACKAGES=LAPACK$<SEMICOLON>BLAS; -DParMETIS_LIBRARIES=/home/tcad/workspace/code/palace/build/lib/libparmetis.a$<SEMICOLON>/home/tcad/workspace/code/palace/build/lib/libmetis.a$<SEMICOLON>/home/tcad/workspace/code/palace/build/lib/libGKlib.a; -DParMETIS_INCLUDE_DIRS=/home/tcad/workspace/code/palace/build/include; -DSuperLUDist_DIR=/home/tcad/workspace/code/palace/build; -DSuperLUDist_REQUIRED_PACKAGES=ParMETIS$<SEMICOLON>METIS$<SEMICOLON>LAPACK$<SEMICOLON>BLAS$<SEMICOLON>MPI
-- ========================== Configuring Palace ==========================
-- PALACE_OPTIONS: -DCMAKE_INSTALL_PREFIX=/home/tcad/workspace/code/palace/build; -DCMAKE_BUILD_TYPE=Release; -DBUILD_SHARED_LIBS=OFF; -DCMAKE_PREFIX_PATH=/home/tcad/workspace/code/palace/build; -DCMAKE_CXX_COMPILER=/usr/bin/c++; -DCMAKE_CXX_FLAGS=; -DPALACE_WITH_OPENMP=OFF; -DPALACE_WITH_SLEPC=ON; -DPALACE_WITH_ARPACK=OFF; -DANALYZE_SOURCES_CLANG_TIDY=OFF; -DANALYZE_SOURCES_CPPCHECK=OFF
-- ======================= Configure stage complete =======================
-- Configuring done (2.0s)
-- Generating done (0.0s)
-- Build files have been written to: /home/tcad/workspace/code/palace/build

Build output:

[  1%] Creating directories for 'gslib'
[  1%] Performing download step (git clone) for 'gslib'
Cloning into 'gslib'...
HEAD is now at 39d1baa Revert "[findpt] No more setup inside findpt_eval (#43)"
[  2%] No update step for 'gslib'
[  3%] Performing patch step for 'gslib'
[  4%] No configure step for 'gslib'
[  5%] No build step for 'gslib'
[  6%] Performing install step for 'gslib'
ar: creating src/libgs.a
a - src/gs.o
a - src/sort.o
a - src/sarray_transfer.o
a - src/sarray_sort.o
a - src/gs_local.o
a - src/fail.o
a - src/crystal.o
a - src/comm.o
a - src/tensor.o
a - src/fcrystal.o
a - src/findpts.o
a - src/findpts_local.o
a - src/obbox.o
a - src/poly.o
a - src/lob_bnd.o
a - src/findpts_el_3.o
a - src/findpts_el_2.o
[  6%] No test step for 'gslib'
[  6%] Completed 'gslib'
[  6%] Built target gslib
[  7%] Creating directories for 'libxsmm'
[  8%] Performing download step (git clone) for 'libxsmm'
Cloning into 'libxsmm'...
HEAD is now at f4ba09230 CI: removed "clxap" and "clxaptrb" (similar to or complements #809).
[  9%] No update step for 'libxsmm'
[  9%] No patch step for 'libxsmm'
[  9%] No configure step for 'libxsmm'
[ 10%] No build step for 'libxsmm'
[ 11%] Performing install step for 'libxsmm'
================================================================================
LIBXSMM main-1.17-4016 (Linux@ubuntu)
--------------------------------------------------------------------------------
GNU Compiler Collection: cc 11.4.0, and c++ 11.4.0
C / C++ target: -msse4.2
Fortran Compiler is disabled or missing: no Fortran interface is built!
--------------------------------------------------------------------------------
--- LIBXSMM build log
================================================================================
LIBXSMM main-1.17-4016 (Linux@ubuntu)
--------------------------------------------------------------------------------
GNU Compiler Collection: cc 11.4.0, and c++ 11.4.0
C / C++ target: -msse4.2
Fortran Compiler is disabled or missing: no Fortran interface is built!
--------------------------------------------------------------------------------
BLAS dependency and fallback is removed!
--------------------------------------------------------------------------------

LIBXSMM installing libraries...
'lib/libxsmmnoblas.so' -> '/home/tcad/workspace/code/palace/build/lib/libxsmmnoblas.so'
'lib/libxsmmnoblas.so.1' -> '/home/tcad/workspace/code/palace/build/lib/libxsmmnoblas.so.1'
'lib/libxsmmnoblas.so.1.17.0' -> '/home/tcad/workspace/code/palace/build/lib/libxsmmnoblas.so.1.17.0'
'lib/libxsmmgen.so' -> '/home/tcad/workspace/code/palace/build/lib/libxsmmgen.so'
'lib/libxsmmgen.so.1' -> '/home/tcad/workspace/code/palace/build/lib/libxsmmgen.so.1'
'lib/libxsmmgen.so.1.17.0' -> '/home/tcad/workspace/code/palace/build/lib/libxsmmgen.so.1.17.0'
'lib/libxsmmext.so' -> '/home/tcad/workspace/code/palace/build/lib/libxsmmext.so'
'lib/libxsmmext.so.1' -> '/home/tcad/workspace/code/palace/build/lib/libxsmmext.so.1'
'lib/libxsmmext.so.1.17.0' -> '/home/tcad/workspace/code/palace/build/lib/libxsmmext.so.1.17.0'
'lib/libxsmm.so' -> '/home/tcad/workspace/code/palace/build/lib/libxsmm.so'
'lib/libxsmm.so.1' -> '/home/tcad/workspace/code/palace/build/lib/libxsmm.so.1'
'lib/libxsmm.so.1.17.0' -> '/home/tcad/workspace/code/palace/build/lib/libxsmm.so.1.17.0'

LIBXSMM installing pkg-config and module files...
'lib/libxsmm-shared.pc' -> '/home/tcad/workspace/code/palace/build/lib/pkgconfig/libxsmm-shared.pc'
'lib/libxsmm.pc' -> '/home/tcad/workspace/code/palace/build/lib/pkgconfig/libxsmm.pc'
'lib/libxsmmext-shared.pc' -> '/home/tcad/workspace/code/palace/build/lib/pkgconfig/libxsmmext-shared.pc'
'lib/libxsmmext.pc' -> '/home/tcad/workspace/code/palace/build/lib/pkgconfig/libxsmmext.pc'
'lib/libxsmmnoblas-shared.pc' -> '/home/tcad/workspace/code/palace/build/lib/pkgconfig/libxsmmnoblas-shared.pc'
'lib/libxsmmnoblas.pc' -> '/home/tcad/workspace/code/palace/build/lib/pkgconfig/libxsmmnoblas.pc'
'lib/libxsmm.env' -> '/home/tcad/workspace/code/palace/build/lib/pkgconfig/libxsmm.env'

LIBXSMM installing interface...
'./include/libxsmm_generator.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_generator.h'
'./include/libxsmm_typedefs.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_typedefs.h'
'./include/libxsmm_fsspmdm.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_fsspmdm.h'
'./include/libxsmm_macros.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_macros.h'
'./include/libxsmm_memory.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_memory.h'
'./include/libxsmm_malloc.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_malloc.h'
'./include/libxsmm_cpuid.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_cpuid.h'
'./include/libxsmm_math.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_math.h'
'./include/libxsmm_sync.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_sync.h'
'./include/utils/libxsmm_intrinsics_x86.h' -> '/home/tcad/workspace/code/palace/build/include/utils/libxsmm_intrinsics_x86.h'
'./include/utils/libxsmm_lpflt_quant.h' -> '/home/tcad/workspace/code/palace/build/include/utils/libxsmm_lpflt_quant.h'
'./include/utils/libxsmm_barrier.h' -> '/home/tcad/workspace/code/palace/build/include/utils/libxsmm_barrier.h'
'./include/utils/libxsmm_timer.h' -> '/home/tcad/workspace/code/palace/build/include/utils/libxsmm_timer.h'
'./include/utils/libxsmm_utils.h' -> '/home/tcad/workspace/code/palace/build/include/utils/libxsmm_utils.h'
'./include/utils/libxsmm_math.h' -> '/home/tcad/workspace/code/palace/build/include/utils/libxsmm_math.h'
'./include/utils/libxsmm_mhd.h' -> '/home/tcad/workspace/code/palace/build/include/utils/libxsmm_mhd.h'
'include/libxsmm_version.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_version.h'
'include/libxsmm_config.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm_config.h'
'include/libxsmm.h' -> '/home/tcad/workspace/code/palace/build/include/libxsmm.h'

LIBXSMM installing header-only...
[ 12%] No test step for 'libxsmm'
[ 13%] Completed 'libxsmm'
[ 13%] Built target libxsmm
[ 14%] Creating directories for 'libCEED'
[ 14%] Performing download step (git clone) for 'libCEED'
Cloning into 'libCEED'...
HEAD is now at f9358d07 Merge pull request #1282 from CEED/jrwrigh/fix-dim-options
[ 15%] No update step for 'libCEED'
[ 16%] Performing patch step for 'libCEED'
[ 17%] No configure step for 'libCEED'
[ 18%] No build step for 'libCEED'
[ 19%] Performing install step for 'libCEED'
make: 'lib' with optional backends: /cpu/self/xsmm/serial /cpu/self/xsmm/blocked
/usr/bin/ld: build/backends/opt/ceed-opt-operator.o: warning: relocation against `CEED_VECTOR_ACTIVE' in read-only section `.text'
/usr/bin/ld: build/interface/ceed-basis.o: relocation R_X86_64_PC32 against symbol `CeedEvalModes' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
gmake[3]: *** [Makefile:552: build/libceed.so] Error 1
make[2]: *** [CMakeFiles/libCEED.dir/build.make:105: extern/libCEED-cmake/src/libCEED-stamp/libCEED-install] Error 2
make[1]: *** [CMakeFiles/Makefile2:152: CMakeFiles/libCEED.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
sebastiangrimberg commented 1 year ago

Very strange indeed. Have you tried just added -fPIC to your CMAKE_C_FLAGS manually? This should probably fix the issue though I'm still not sure why it isn't working as is.

EMinsight commented 1 year ago

I added -fPIC manually to CMAKE_C_FLAGS in ExternalLibCEED.cmake, and it works.

I failed to build the master through Spack, it said that cannot find -lxsmm while building mfem.

EMinsight commented 1 year ago

After taking a look at the makefile in libCEED. I found the makefile cannot get the correct CC_VENDOR, it is empty. Maybe this is the root cause.

EMinsight commented 1 year ago

More update:

The CMake passes the CC=/usr/bin/cc to the makefile and the makefile gets the empty CC_VENDOR.

sebastiangrimberg commented 1 year ago

Aha, good find. It looks like maybe the /usr/bin/cc being detected by CMake isn't playing nicely with libCEED's detection for some reason.

What's the output of /usr/bin/cc --version? Could you also try gcc --version?

You can also try instead using CMAKE_C_COMPILER=gcc and CMAKE_CXX_COMPILER=g++.

I will have a fix for the Spack + libCEED build as part of #97, sorry I forgot about that issue.

EMinsight commented 1 year ago

/usr/bin/cc and /usr/bin/gcc have same version 11.4.0.

sebastiangrimberg commented 1 year ago

Yes but the libCEED Makefile is going to parse the output to define CC_VENDOR. See this line: https://github.com/CEED/libCEED/blob/533adc1e28eea3d12734d65713e13d9fda16729c/Makefile#L99. So, it will look for any word in the output which matches any of gcc clang icc icc_orig oneAPI XL emcc. Ideally only one (gcc) should match in your case.

EMinsight commented 1 year ago

My output is cc which is not included in the keyword lists.

sebastiangrimberg commented 1 year ago

Ah, OK. This I can also reproduce on a RHEL setup:

$ cc --version
cc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-17)
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The solution here in the short term for you is to provice CMAKE_C_COMPILER=gcc and CMAKE_CXX_COMPILER=g++. I will submit a bug fix to libCEED to detect GCC as gcc.

sebastiangrimberg commented 1 year ago

See https://github.com/CEED/libCEED/pull/1368. This update will get fixed as part of #97. For now I would just make sure you defined the CMake compiler variables to gcc and g++. Thanks for your help in diagnosing this issue!

EMinsight commented 1 year ago

Thanks, you can close this issue as you need.

EMinsight commented 1 year ago

This is my cc --version output:

$ cc --version
cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The CEED/LibCEED#1386 seems not working for my setups.

sebastiangrimberg commented 1 year ago

I see, it does seem that the libCEED fix won't work for Ubuntu. For RHEL, it would fix the issue:

$ cc --version
cc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-15)
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

For your case I would advice just installing GCC (sudo apt-get install build-essential) and using that instead of relying on cc/c++, either by export CC=gcc; export CXX=g++ before calling CMake, or by passing -DCMAKE_C_COMPILER=gcc; -DCMAKE_CXX_COMPILER=g++ to the cmake command. Either of these approaches should resolve your issue.

EMinsight commented 1 year ago

Thanks.