search

CASL / Futility

VERA Fortran Utilities
Other
44 stars 20 forks source link

Make work with updated TriBITS and Trilinos matching CASL VERA #221

Closed bartlettroscoe closed 3 years ago

bartlettroscoe commented 4 years ago

This goes along with the fixes to Trilinos in https://github.com/CASL/Trilinos/pull/1.

Without these changes, Futrility will not build with that updated version of Trilinos.

bartlettroscoe commented 4 years ago

@bscollin, I updated the TriBITS snapshot so this and PR https://github.com/CASL/Trilinos/pull/1 should be good to go.

Let me know if you have any trouble with this.

bscollin commented 4 years ago

Minor issue wit this is that the docker looks like it needs to be updated to a more modern version of cmake. Also, I've made a repo: github.com/CASL/Trilinos so that we don't need to checkout a specific SHA1 anymore. The second should be easy to do, the first will be more work.

bartlettroscoe commented 4 years ago

Below are my notes for debugging and addressing this issue. Also, this is info on how I can reproduce Futility builds for the next time I need to do this. Details are given below in "Detailed notes". In short, I used CMake 3.12.2 and GNU 7.2.0 compilers to build Futility on my local machine. The test results where:

100% tests passed, 0 tests failed out of 57

Subproject Time Summary:
Futility    =  15.37 sec*proc (57 tests)

Label Time Summary:
BASIC         =  15.37 sec*proc (57 tests)
CONTINUOUS    =  15.37 sec*proc (57 tests)
HEAVY         =  15.37 sec*proc (57 tests)
NIGHTLY       =  15.37 sec*proc (57 tests)

(When did the testing categories get added as test labels? That is really nice!)

TASKS:

DETAILED NOTES: (click to expand) . **(2020/01/29)** Cloning the Futility, Trilinos, and TriBITS repos from the github.com/CASL site. On the machine 'ews00232': ``` $ cd /scratch/rabartl/Futility.base/ $ git clone git@github.com:CASL/Futility.git $ cd Futility/ $ git clone git@github.com:CASL/TriBITS.git $ git clone git@github.com:CASL/Trilinos.git $ $ echo ". TriBITS Trilinos " > .gitdist $ gitdist remote rename origin casl-github ``` This gives the repo state: ``` $ gitdist-show-full-repo-state Repo versions: *** Base Git Repo: Futility e74720e [Wed Jan 15 11:40:31 2020 -0500] Fix indentations (#220) *** Git Repo: TriBITS 91aaa7a [Mon Nov 11 14:33:26 2019 -0500] Merge branch '6124-CheckinTest-update' into 'master' *** Git Repo: Trilinos 3cc9130 [Wed Jan 3 15:28:13 2018 -0700] Fixed bug where the wrong basis was being constructed (#2113) Repo branch status: ------------------------------------------------------------------ | ID | Repo Dir | Branch | Tracking Branch | C | M | ? | |----|-----------------|--------|--------------------|---|---|---| | 0 | Futility (Base) | master | casl-github/master | | | 1 | | 1 | TriBITS | master | casl-github/master | | | | | 2 | Trilinos | master | casl-github/master | | | | ------------------------------------------------------------------ Repo remotes: *** Base Git Repo: Futility casl-github git@github.com:CASL/Futility.git (push) *** Git Repo: TriBITS casl-github git@github.com:CASL/TriBITS.git (push) *** Git Repo: Trilinos casl-github git@github.com:CASL/Trilinos.git (push) ``` Looks like that version of Trilinos is too old. The version of Trilinos that I merged in for the CASL VERA update in was from: ``` 7396a09 "Merge pull request #5126 from trilinos/5042_deprecated_row_add" Author: Mark Hoemmen Date: Wed May 8 09:20:33 2019 -0600 (9 months ago) ``` ... I sent Ben an email and he updated the CASL/Trilinos 'master' branch. Now the repo state is: ``` $ gitdist-show-full-repo-state Repo versions: *** Base Git Repo: Futility e74720e [Wed Jan 15 11:40:31 2020 -0500] Fix indentations (#220) *** Git Repo: TriBITS 91aaa7a [Mon Nov 11 14:33:26 2019 -0500] Merge branch '6124-CheckinTest-update' into 'master' *** Git Repo: Trilinos 170f8f2 [Fri May 10 09:14:30 2019 -0600] Make work when TriBITS project is not Trilinos (PHI-5294) Repo branch status: ------------------------------------------------------------------ | ID | Repo Dir | Branch | Tracking Branch | C | M | ? | |----|-----------------|--------|--------------------|---|---|---| | 0 | Futility (Base) | master | casl-github/master | | | 1 | | 1 | TriBITS | master | casl-github/master | | | | | 2 | Trilinos | master | casl-github/master | | | | ------------------------------------------------------------------ Repo remotes: *** Base Git Repo: Futility casl-github git@github.com:CASL/Futility.git (push) *** Git Repo: TriBITS casl-github git@github.com:CASL/TriBITS.git (push) *** Git Repo: Trilinos casl-github git@github.com:CASL/Trilinos.git (push) ``` Now to try a configure. First, I need an env. Let's try the env: ``` $ cat load-env.sh module purge module load sems-env module load sems-git/2.10.1 module load sems-cmake/3.12.2 module load sems-ninja_fortran/1.8.2 module load sems-gcc/7.2.0 module load sems-openmpi/1.10.1 ``` Let's try the configure script: ``` $ cat do-configure #!/bin/bash if [[ -e CMakeCache.txt ]] ; then rm -r CMake* fi cmake \ -DFutility_TRIBITS_DIR:STRING=TriBITS/tribits \ -DFutility_ENABLE_TESTS=ON \ "$@" \ /scratch/rabartl/Futility.base/Futility ``` Now to try to configure: ``` $ ssh ews00232 $ cd /scratch/rabartl/Futility.base/BUILDS/GCC-7.2.0/MPI_DEBUG/ $ . load-env.sh $ module list Currently Loaded Modulefiles: 1) sems-env 3) sems-cmake/3.12.2 5) sems-gcc/7.2.0 2) sems-git/2.10.1 4) sems-ninja_fortran/1.8.2 6) sems-openmpi/1.10.1 $ time ./do-configure &> configure.out real 0m26.683s user 0m14.147s sys 0m9.150s ``` That produced the configure error: ``` CMake Error at Trilinos/packages/kokkos/CMakeLists.txt:102 (set_kokkos_srcs): Unknown CMake command "set_kokkos_srcs". ``` I then added the Futrility commits: ``` *** Base Git Repo: Futility 0a32b01 "Allow override of _TRIBITS_DIR in the user cache" Author: Roscoe A. Bartlett Date: Wed Jan 29 13:58:16 2020 -0700 (2 minutes ago) M cmake/Settings_Common.cmake c186abe "Ignore TriBITS and make ignores more direct" Author: Roscoe A. Bartlett Date: Wed Jan 29 13:57:46 2020 -0700 (2 minutes ago) M .gitignore 715eebf "Add ProjectCompilerPostConfig.cmake to include the one from Trilinos (PHI-5294)" Author: Roscoe A. Bartlett Date: Fri May 10 09:19:23 2019 -0600 (19 hours ago) A cmake/ProjectCompilerPostConfig.cmake ``` The configure then showed the warning: ``` CMake Warning (dev) at Trilinos/cmake/ProjectCompilerPostConfig.cmake:1 (IF): if given arguments: "ON" An argument named "ON" appears in a conditional statement. Policy CMP0012 is not set: if() recognizes numbers and boolean constants. Run "cmake --help-policy CMP0012" for policy details. Use the cmake_policy command to set the policy and suppress this warning. Call Stack (most recent call first): cmake/ProjectCompilerPostConfig.cmake:1 (include) TriBITS/tribits/core/package_arch/TribitsGlobalMacros.cmake:1952 (INCLUDE) TriBITS/tribits/core/package_arch/TribitsProjectImpl.cmake:192 (TRIBITS_SETUP_ENV) TriBITS/tribits/core/package_arch/TribitsProject.cmake:93 (TRIBITS_PROJECT_IMPL) cmake/Configure_Common.cmake:11 (TRIBITS_PROJECT) CMakeLists.txt:20 (INCLUDE) ``` No idea why Futility is triggering this warning when Trilinos and CASL VERA does not with the same file. But the fix is trivial: ``` *** Git Repo: Trilinos 8fb5134 "Fixed bad if statement logic (CASL PHI #6249)" Author: Roscoe A. Bartlett Date: Wed Jan 29 13:55:24 2020 -0700 (18 hours ago) M cmake/ProjectCompilerPostConfig.cmake ``` I then configured again and got a bunch of build warnings of the same type from Trilinos that I fixed with the Trilinos commits: ``` *** Git Repo: Trilinos 0ade85e "Fix invalid if statement variable usage" Author: Roscoe A. Bartlett Date: Wed Jan 29 17:17:25 2020 -0700 (15 hours ago) M packages/kokkos-kernels/CMakeLists.txt M packages/kokkos/cmake/kokkos_settings.cmake ``` And there was one more from Futrility that I fixed: ``` *** Base Git Repo: Futility 1992c03 "Fix bad if statement varaible usage" Author: Roscoe A. Bartlett Date: Wed Jan 29 17:18:33 2020 -0700 (15 hours ago) M unit_tests/testDBC/CMakeLists.txt ``` I then snapshotted this version of TriBITS (from CASL VERA) into Futrility and the final commits were: ``` *** Base Git Repo: Futility ccd5751 "Automatic snapshot commit from tribits at e96a835" Author: Roscoe A. Bartlett Date: Wed Jan 29 20:38:10 2020 -0700 (12 hours ago) ... 1992c03 "Fix bad if statement varaible usage" Author: Roscoe A. Bartlett Date: Wed Jan 29 17:18:33 2020 -0700 (15 hours ago) M unit_tests/testDBC/CMakeLists.txt 368c1e3 "Allow override of _TRIBITS_DIR in the user cache" Author: Roscoe A. Bartlett Date: Wed Jan 29 13:58:16 2020 -0700 (18 hours ago) M cmake/Settings_Common.cmake f6b32aa "Ignore TriBITS and make ignores more direct" Author: Roscoe A. Bartlett Date: Wed Jan 29 13:57:46 2020 -0700 (18 hours ago) M .gitignore 715eebf "Add ProjectCompilerPostConfig.cmake to include the one from Trilinos (PHI-5294)" Author: Roscoe A. Bartlett Date: Fri May 10 09:19:23 2019 -0600 (19 hours ago) A cmake/ProjectCompilerPostConfig.cmake *** Git Repo: TriBITS *** Git Repo: Trilinos 0ade85e "Fix invalid if statement variable usage" Author: Roscoe A. Bartlett Date: Wed Jan 29 17:17:25 2020 -0700 (15 hours ago) M packages/kokkos-kernels/CMakeLists.txt M packages/kokkos/cmake/kokkos_settings.cmake 8fb5134 "Fixed bad if statement logic (CASL PHI #6249)" Author: Roscoe A. Bartlett Date: Wed Jan 29 13:55:24 2020 -0700 (18 hours ago) M cmake/ProjectCompilerPostConfig.cmake ``` Now final configure with the confiugre script: ``` $ cat do-configure #!/bin/bash if [[ -e CMakeCache.txt ]] ; then rm -r CMake* fi cmake \ -DFutility_ENABLE_TESTS=ON \ "$@" \ /scratch/rabartl/Futility.base/Futility ``` Doing the configure, build and testing: ``` $ ssh ews00232 $ cd /scratch/rabartl/Futility.base/BUILDS/GCC-7.2.0/MPI_DEBUG/ $ . load-env.sh $ module list Currently Loaded Modulefiles: 1) sems-env 3) sems-cmake/3.12.2 5) sems-gcc/7.2.0 2) sems-git/2.10.1 4) sems-ninja_fortran/1.8.2 6) sems-openmpi/1.10.1 $ time ./do-configure &> configure.out real 0m25.241s user 0m13.773s sys 0m8.662s $ time make -j8 &> make.out real 11m9.563s user 46m58.397s sys 4m19.805s $ time ctest -j8 &> ctest.out real 0m5.562s user 0m12.410s sys 0m0.761s $ grep -A 100 "failed out of" ctest.out 100% tests passed, 0 tests failed out of 57 Subproject Time Summary: Futility = 15.37 sec*proc (57 tests) Label Time Summary: BASIC = 15.37 sec*proc (57 tests) CONTINUOUS = 15.37 sec*proc (57 tests) HEAVY = 15.37 sec*proc (57 tests) NIGHTLY = 15.37 sec*proc (57 tests) Total Test time (real) = 5.56 sec ``` This showed in the configure output: ``` Final set of enabled packages: Kokkos Teuchos KokkosKernels RTOp Epetra Triutils EpetraExt Tpetra TrilinosSS Thyra Xpetra AztecOO Amesos Ifpack Belos ShyLU_Node Amesos2 Anasazi Ifpack2 Stratimikos MueLu Futility 22 Final set of enabled SE packages: KokkosCore KokkosContainers KokkosAlgorithms Kokkos TeuchosCore TeuchosParser TeuchosParameterList TeuchosComm TeuchosNumerics TeuchosRemainder TeuchosKokkosCompat TeuchosKokkosComm Teuchos KokkosKernels RTOp Epetra Triutils EpetraExt TpetraClassic TpetraTSQR TpetraCore Tpetra TrilinosSS ThyraCore ThyraEpetraAdapters ThyraEpetraExtAdapters ThyraTpetraAdapters Thyra Xpetra AztecOO Amesos Ifpack Belos ShyLU_NodeHTS ShyLU_NodeTacho ShyLU_Node Amesos2 Anasazi Ifpack2 Stratimikos MueLu Futility 42 ``` ``` Final set of enabled TPLs: BLAS LAPACK DLlib 3 ``` ``` -- CMAKE_BUILD_TYPE='DEBUG' ``` ``` -- The C compiler identification is GNU 7.2.0 -- Check for working C compiler: /projects/sems/install/rhel7-x86_64/sems/compiler/gcc/7.2.0/base/bin/gcc -- Check for working C compiler: /projects/sems/install/rhel7-x86_64/sems/compiler/gcc/7.2.0/base/bin/gcc -- works ... -- CMAKE_C_COMPILER_ID='GNU' -- CMAKE_C_COMPILER_VERSION='7.2.0' ``` ``` -- The CXX compiler identification is GNU 7.2.0 -- Check for working CXX compiler: /projects/sems/install/rhel7-x86_64/sems/compiler/gcc/7.2.0/base/bin/g++ -- Check for working CXX compiler: /projects/sems/install/rhel7-x86_64/sems/compiler/gcc/7.2.0/base/bin/g++ -- works ... -- CMAKE_CXX_COMPILER_ID='GNU' -- CMAKE_CXX_COMPILER_VERSION='7.2.0' ``` ``` -- The Fortran compiler identification is GNU 7.2.0 -- Check for working Fortran compiler: /projects/sems/install/rhel7-x86_64/sems/compiler/gcc/7.2.0/base/bin/gfortran -- Check for working Fortran compiler: /projects/sems/install/rhel7-x86_64/sems/compiler/gcc/7.2.0/base/bin/gfortran -- works ... -- Checking whether /projects/sems/install/rhel7-x86_64/sems/compiler/gcc/7.2.0/base/bin/gfortran supports Fortran 90 -- Checking whether /projects/sems/install/rhel7-x86_64/sems/compiler/gcc/7.2.0/base/bin/gfortran supports Fortran 90 -- yes ``` ``` Processing enabled TPL: BLAS (enabled by TeuchosNumerics, disable with -DTPL_ENABLE_BLAS=OFF) ... -- TPL_BLAS_LIBRARIES='/usr/lib64/libblas.so' ``` ``` Processing enabled TPL: LAPACK (enabled by TeuchosNumerics, disable with -DTPL_ENABLE_LAPACK=OFF) ... -- TPL_LAPACK_LIBRARIES='/usr/lib64/liblapack.so;/usr/lib64/libblas.so' ``` ``` Processing enabled TPL: DLlib (enabled explicitly, disable with -DTPL_ENABLE_DLlib=OFF) ... -- TPL_DLlib_LIBRARIES='/usr/lib64/libdl.so' ``` ``` Processing enabled package: Futility (Libs, Tests, Examples) -- Enabling BLAS Routines -- Enabling Design by Contract Checking ``` That is pretty simple and looks good to me.
bartlettroscoe commented 4 years ago

FYI: using the configure script:

$ cat do-configure
#!/bin/bash

if [[ -e CMakeCache.txt ]] ; then
  rm -r CMake*
fi

cmake \
-GNinja \
-DFutility_ENABLE_TESTS=ON \
"$@" \
/scratch/rabartl/Futility.base/Futility

I also tried this with ninja-fortran 1.8.2 from the Kitware fork, and it worked as well, but for some reason it build very slowly. The ninja build should use more parallelism and build faster but for some reason it only looked like it used one core to build. Not sure what that is about but that is a same. We see noticeable improvements in build times with Trilinos (especially on rebuilds due to faster dependency rule evaluation).

stimpsonsg commented 4 years ago

Hmmm...does something need to be updated on the Travis CI side?

bscollin commented 4 years ago

Yes... I told you that... waiting on Mark

stimpsonsg commented 4 years ago

ok

bartlettroscoe commented 4 years ago

@bscollin,

I just pushed the commit bfae4c598ce40291d5300517675ef9fca2b60128 which sets Tpetra_INST_INT_INT=OFF. That should result in:

Processing ETI support: TpetraCore
-- TpetraCore: Processing ETI / test support
-- Enabled Scalar types:        long long|double
-- Enabled LocalOrdinal types:  int
-- Enabled GlobalOrdinal types: long long
-- Enabled Node types:          Kokkos::Compat::KokkosSerialWrapperNode
-- Set of enabled types, before exclusions: S={long long} N={Kokkos::Compat::KokkosSerialWrapperNode} LO={int} GO={long long};S={double} N={Kokkos::Compat::KokkosSerialWrapperNode} LO={int} GO={long long}

In other words, that should cut the number of Tpetra instantiations form 6 to 2.