search

GEOS-DEV / thirdPartyLibs

Repository to build the GEOSX third party libraries
3 stars 12 forks source link

Issue compiling Trilinos #136

Closed ishovkun closed 3 years ago

ishovkun commented 3 years ago

I am unable to compile Trilinos.

My host-config: https://pastebin.com/mBuvcBVy Trilinos build log (up to the first errors): https://pastebin.com/Apckm1ju Log of the overall cmake run on TPL repo: https://pastebin.com/mF0ZnViy

Additional details: Platform: Manjaro Linux Compiler suite: gcc 10.2

Let me know should I post additional details.

ishovkun commented 3 years ago

Pinging @klevzoff and @corbett5.

corbett5 commented 3 years ago

Weird I've never seen that error before. Can you paste the contents of packages/kokkos/cmake/KokkosCore_config.h.in, packages/kokkos/core/cmake/KokkosCore_config.h.in, and build-linux-gcc-release/trilinos/src/trilinos-build/KokkosCore_config.h? The first two files should be in build-linux-gcc-release/trilinos/src/trilinos-src/ or something like that.

klevzoff commented 3 years ago

Never seen this either, but other people have: https://github.com/spack/spack/issues/16308 Apparently cmake's configure_file adds those backslashes for no reason sometimes?

corbett5 commented 3 years ago

Wow, nice find! Since they're using gcc9 that might be the issue, although it seems that this should be completely orthogonal to the compiler.

ishovkun commented 3 years ago

The contents of packages/kokkos/core/cmake/KokkosCore_config.h.in:

/* The trivial 'src/build_common.sh' creates a config
 * that must stay in sync with this file.
 */
#cmakedefine KOKKOS_FOR_SIERRA

#if !defined(KOKKOS_FOR_SIERRA)

#if !defined(KOKKOS_MACROS_HPP) || defined(KOKKOS_CORE_CONFIG_H)
#error "Don't include KokkosCore_config.h directly; include Kokkos_Macros.hpp instead."
#else
#define KOKKOS_CORE_CONFIG_H
#endif

#cmakedefine KOKKOS_ENABLE_CUDA
#cmakedefine KOKKOS_ENABLE_OPENMP
#cmakedefine KOKKOS_ENABLE_PTHREAD
#cmakedefine KOKKOS_ENABLE_QTHREADS
#cmakedefine KOKKOS_ENABLE_SERIAL
#cmakedefine KOKKOS_ENABLE_Winthread

#cmakedefine KOKKOS_ENABLE_HWLOC
#cmakedefine KOKKOS_ENABLE_HBWSPACE
#cmakedefine KOKKOS_ENABLE_LIBRT

#cmakedefine KOKKOS_ENABLE_DEBUG
#cmakedefine KOKKOS_ENABLE_DEBUG_BOUNDS_CHECK
#cmakedefine KOKKOS_ENABLE_DEBUG_DUALVIEW_MODIFY_CHECK
#cmakedefine KOKKOS_ENABLE_PROFILING
#cmakedefine KOKKOS_ENABLE_PROFILING_LOAD_PRINT

#cmakedefine KOKKOS_ENABLE_AGGRESSIVE_VECTORIZATION

#ifdef KOKKOS_ENABLE_CUDA

#cmakedefine KOKKOS_ENABLE_CUDA_LDG_INTRINSIC

// mfh 16 Sep 2014: If passed in on the command line, that overrides
// any value of KOKKOS_USE_CUDA_UVM here.  Doing this should prevent build
// warnings like this one:
//
// packages/kokkos/core/src/KokkosCore_config.h:13:1: warning: "KOKKOS_USE_CUDA_UVM" redefined
//
// At some point, we should edit the test-build scripts in
// Trilinos/cmake/ctest/drivers/perseus/, and take
// -DKOKKOS_USE_CUDA_UVM from the command-line arguments there.  I
// hesitate to do that now, because I'm not sure if all the files are
// including KokkosCore_config.h (or a header file that includes it) like
// they should.
#ifndef KOKKOS_USE_CUDA_UVM
#cmakedefine KOKKOS_USE_CUDA_UVM
#endif

#cmakedefine KOKKOS_ENABLE_CUDA_RELOCATABLE_DEVICE_CODE

#cmakedefine KOKKOS_ENABLE_CUDA_LAMBDA

#endif

#cmakedefine KOKKOS_IMPL_CUDA_CLANG_WORKAROUND

#ifndef __CUDA_ARCH__
#cmakedefine KOKKOS_ENABLE_ISA_X86_64
#cmakedefine KOKKOS_ENABLE_ISA_KNC
#cmakedefine KOKKOS_ENABLE_ISA_POWERPCLE
#endif

#cmakedefine KOKKOS_ARCH_ARMV80 1
#cmakedefine KOKKOS_ARCH_ARMV81 1
#cmakedefine KOKKOS_ARCH_ARMV8_THUNDERX 1
#cmakedefine KOKKOS_ARCH_AVX 1
#cmakedefine KOKKOS_ARCH_AVX2 1
#cmakedefine KOKKOS_ARCH_AVX512MIC 1
#cmakedefine KOKKOS_ARCH_AVX512XEON 1
#cmakedefine KOKKOS_ARCH_KNC 1
#cmakedefine KOKKOS_ARCH_POWER8 1
#cmakedefine KOKKOS_ARCH_POWER9 1
#cmakedefine KOKKOS_ARCH_KEPLER 1
#cmakedefine KOKKOS_ARCH_KEPLER30 1
#cmakedefine KOKKOS_ARCH_KEPLER32 1
#cmakedefine KOKKOS_ARCH_KEPLER35 1
#cmakedefine KOKKOS_ARCH_KEPLER37 1
#cmakedefine KOKKOS_ARCH_MAXWELL 1
#cmakedefine KOKKOS_ARCH_MAXWELL50 1
#cmakedefine KOKKOS_ARCH_MAXWELL52 1
#cmakedefine KOKKOS_ARCH_MAXWELL53 1
#cmakedefine KOKKOS_ARCH_PASCAL 1
#cmakedefine KOKKOS_ARCH_PASCAL60 1
#cmakedefine KOKKOS_ARCH_PASCAL61 1
#cmakedefine KOKKOS_ARCH_VOLTA70 1

// TODO: These are currently not used in Kokkos.  Should they be removed?
#cmakedefine KOKKOS_ENABLE_MPI
#cmakedefine KOKKOS_ENABLE_CUSPARSE

// TODO: No longer options in Kokkos.  Need to be removed.
#cmakedefine KOKKOS_USING_DEPRECATED_VIEW
#cmakedefine KOKKOS_ENABLE_CXX11

#endif // !defined(KOKKOS_FOR_SIERRA)
ishovkun commented 3 years ago

I do not see KokkosCore_config.h.in in packages/kokkos/cmake, but I do see a KokkosConfig.cmake.in. Do you want to see that?

ishovkun commented 3 years ago

Here is the contents of build-linux-gcc-release/trilinos/src/trilinos-build/KokkosCore_config.h: 

/* ---------------------------------------------
Makefile constructed configuration:
Mon 19 Oct 2020 11:38:02 AM PDT
----------------------------------------------*/
\#if !defined(KOKKOS_MACROS_HPP) || defined(KOKKOS_CORE_CONFIG_H)
\#error "Do not include KokkosCore_config.h directly; include Kokkos_Macros.hpp instead."
\#else
\#define KOKKOS_CORE_CONFIG_H
\#endif
/* Execution Spaces */
\#define KOKKOS_ENABLE_SERIAL
/* General Settings */
\#define KOKKOS_ENABLE_CXX11
\#define KOKKOS_ENABLE_PROFILING
\#define KOKKOS_ENABLE_DEPRECATED_CODE
/* Optimization Settings */
/* Cuda Settings */
corbett5 commented 3 years ago

Actually they're using cmake@3.17 which is more likely to be the issue.

ishovkun commented 3 years ago

I use cmake-3.18.1

corbett5 commented 3 years ago

Yeah sorry I was looking at a newer version of Trilinos so packages/kokkos/cmake/KokkosCore_config.h.in doesn't exist. Can you try and install cmake 3.14.5 and see if that fixes it?

ishovkun commented 3 years ago

I'll try.

corbett5 commented 3 years ago

Once you've configured just do make trilinos again so you don't have to wait around for the other libraries to build.

klevzoff commented 3 years ago

There's some indication that it might have something to with version of make 4.3: Debian added a custom patch earlier this year to work around it (but it's for an older trilinos release and obviously not backported into mainstream) https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=961708

On my system make is 4.2.1 and I'm not seeing the problem. Just to confirm, what is make --version for you @ishovkun ?

ishovkun commented 3 years ago

my Make is 4.3

klevzoff commented 3 years ago

In fact looking at build-linux-gcc-release/trilinos/src/trilinos/packages/kokkos/Makefile.kokkos you can see where those backslashes come from. It might be the case that older make would interpret them as escape symbols and newer does not. I'm not sure why is it using that Makefile at all rather than fully relying on CMake-generated build system.

ishovkun commented 3 years ago

@corbett5 Using CMake-3.14.5 did not help: the build still fails with the same type of errors:

/home/ishovkun/dev/thirdPartyLibs/build-linux-gcc-release/trilinos/src/trilinos-build/KokkosCore_config.h:5:1: error: stray ‘\’ in program
    5 | \#if !defined(KOKKOS_MACROS_HPP) || defined(KOKKOS_CORE_CONFIG_H)                                                                                                                                          
      | ^
ishovkun commented 3 years ago

@klevzoff So do you suggest to use an older make?

corbett5 commented 3 years ago

Yeah I'd give that a try.

It's funny because it seems they're aware of some issue with make 4.3 https://github.com/trilinos/Trilinos/blob/4285cf9c76dcad84dcea1afe565006a6ae136bbe/packages/kokkos/Makefile.kokkos#L446

klevzoff commented 3 years ago

So it looks like they also fixed it in upstream. If it did make it into 13.0 release, we're going to switch to that hopefully soon enough and the problem will be fixed. Otherwise, we might need to make our own patch.

In the meantime, trying a pre-4.3 make would be a workaround, if feasible without breaking your whole system.

ishovkun commented 3 years ago

@klevzoff Which one do you suggest to go with? https://ftp.gnu.org/gnu/make/

klevzoff commented 3 years ago

@klevzoff Which one do you suggest to go with? https://ftp.gnu.org/gnu/make/

4.2.1 definitely works and is the closest to the one you have, so I'd suggest that perhaps

ishovkun commented 3 years ago

So I compiled my own make (4.2.1) and used it during /path/to/make-4.2.1/make trilinos. I get literally the same errors. Maybe I should cmake to this make instance prior to pyhon2 configure script invocation?

corbett5 commented 3 years ago

I think you'd need to put it in your path so that which make returns your make 4.2.1 and not the system make 4.3. This might be tricky.

ishovkun commented 3 years ago

OK, so I did 

export PATH=/path/to/make-4.2.1:$PATH

I confirm that Trilinos compiles. Thank you @corbett5 and @klevzoff so much for your help. I will confirm in the chat if I am able to build the whole TPL repo.