CMake Build

[islas]

TYPE: new feature

KEYWORDS: CMake, build, make

SOURCE: internal

DESCRIPTION OF CHANGES: Problem: Current WPS build system has similar complexity issues to the WRF build system, but also is tightly coupled to the WRF build system relying on specific object files directly instead of a library.

Solution: A redesign of the build system from the ground up, maintaining the interfacing feel and knowledge accumulated in arch/configure.defaults. This will be a work in progress as gaps are identified in reproducing the full functionality of the makefile build system. Additionally removes the strong coupling of object files coming from WRF, and instead relies on static libraries

LIST OF MODIFIED FILES: A arch/configure_reader.py A cleanCMake.sh A cmake/modules/FindJasper.cmake A cmake/modules/FindnetCDF-Fortran.cmake A cmake/modules/FindnetCDF.cmake A cmake/README A cmake/template/arch_config.cmake A cmake/template/WPSConfig.cmake.in A cmake/wrf_case_setup.cmake A cmake/wrf_get_version.cmake A CMakeLists.txt A compile_new A configure_new A external/CMakeLists.txt A geogrid/CMakeLists.txt A geogrid/src/CMakeLists.txt A metgrid/CMakeLists.txt A metgrid/src/CMakeLists.txt A ungrib/CMakeLists.txt A ungrib/src/CMakeLists.txt A ungrib/src/ngl/CMakeLists.txt A ungrib/src/ngl/g2/CMakeLists.txt A ungrib/src/ngl/w3/CMakeLists.txt A util/CMakeLists.txt A util/src/CMakeLists.txt M .gitignore M arch/configure.defaults M external/.gitignore M README

TESTS CONDUCTED:

1. In various instances this build is faster and more reliable with meaningful diagnostics when errors occur RELEASE NOTE: Introduction of a CMake build system

[islas]

This is the accompanying WPS build, as a major update to the WRF build system requires changes in WPS as well

[islas]

The major changes in configure.defaults was to make sure configurations had the same listing of important flags. Before the edits, configs had wildly varying sets of variables and that inconsistency was making the generic arch/configure_reader.py (which is shared with WRF) more difficult than it ought to be.

[mgduda]

Working on Derecho and choosing options 1, n, n, I get the following error:

-- The C compiler identification is Intel 2021.10.0.20230609
-- Cray Programming Environment 2.7.23 C
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /glade/u/apps/derecho/23.09/spack/opt/spack/ncarcompilers/1.0.0/oneapi/2023.2.1/6q5s/bin/icc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is IntelLLVM 2023.2.0
-- Cray Programming Environment 2.7.23 CXX
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /glade/u/apps/derecho/23.09/spack/opt/spack/ncarcompilers/1.0.0/oneapi/2023.2.1/6q5s/bin/icpx - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The Fortran compiler identification is Intel 2021.10.0.20230609
-- Cray Programming Environment 2.7.23 Fortran
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /glade/u/apps/derecho/23.09/spack/opt/spack/ncarcompilers/1.0.0/oneapi/2023.2.1/6q5s/bin/ifort - skipped
-- No WRF_DIR or WRF_ROOT provided, skipping targets that rely on WRF
-- Setting project version to 4.4
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11")  
-- Found PNG: /glade/u/apps/derecho/23.09/opt/view/lib64/libpng.so (found version "1.6.39") 
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2") 
-- Found Jasper: /glade/u/home/wrfhelp/UNGRIB_LIBRARIES/lib/libjasper.a (found suitable version "1.900.1", required range is "1.900.1...1.900.29") 
CMake Error at CMakeLists.txt:108 (if):
  if given arguments:

    "OFF" "AND"

  Unknown arguments specified

-- Configuring incomplete, errors occurred!

[islas]

Woops, forgot to retest with WRF not enabled. Fix coming in shortly

[mgduda]

The last commit resolves the issue I was seeing. Actually, my intent wasn't to only compile ungrib, so perhaps I'm misunderstanding the option:

[EXT] Build external? Default [N] [y/N] :

What does this control, or what does it mean to build external?

[islas]

That allows for building the external libs (zlib, libpng, and jasper) from the ones under the external/ folder.

The equivalent configure command should be --build-grib2-libs

metgrid and geogrid will only be built if a path is provided to WRF (via cmake-style for packages which includes env WRF_DIR support). The WRF build currently has to be cmake-based as well, but if there is strong need we can change that.

[mgduda]

I think it would be nice if users didn't need to specify the path to the compiled WRF code. I can appreciate that the situation is a bit different when WRF is built out-of-source. But historically, I think most users have relied on the WPS to detect the compiled WRF code in ../WRF and use that without the need to set environment variables, etc.

[mgduda]

I would also propose that the default for

[DM] Use MPI?    Default [Y] [Y/n] :

be n, as we most often build the WPS executables without MPI. It's typically only in cases of very large domains where the ability to make use of the aggregate memory across multiple nodes is useful.

[islas]

I can go ahead and add the check for ../WRF - would you want the behavior to be:

1. automatically try to find WRF and continue with just ungrib if not found
2. ask if met/geo are wanted which implies wanting WRF, then fail if WRF not found

(2) seems good, though it does add an extra arg to controlling met/geo compilation where you request them be build + a WRF dir if you want an alternate location.

Though I suppose almost everything works that way, save for that alternate locations are to search in locations that are not the system standard locations vs WRF standard.

[mgduda]

What we do currently is: 1) If the WRF_DIR environment variable is defined, use it to determine the location of the compiled WRF code 2) Otherwise, look in ../WRF (or a few other directories that are outdated)

However, if the --nowrf option has been specified to the configure script, we skip all of the above and assume that the user doesn't intend to compile either geogrid or metgrid (which depend on WRF).

Being able to do something similar in the CMake-based build system would be nice.

[islas]

Petition to not fail if --no-wrf and no WRF dir is found, that way if you are compiling MPAS you don't need to specify more, and now --no-wrf is for if you do happen to have a WRF build next to WPS but for MPAS and don't want metgrid/geogrid.

I'd like to have it the option also exposed from the prompt queries instead of - style to keep the config script generic between WRF and WPS

edit: It would fail if a valid WRF dir exists but isn't compiled properly

[mgduda]

Agreed regarding no failure if --no-wrf is specified and no WRF directory is found. Also, I think it would be fine to prompt for something like Detect compiled WRF (needed by geogrid and metgrid)? Default [Y] [Y/n]: or similar.

[mgduda]

@islas This prompt seems a bit cryptic:

[EXT] Build external? Default [N] [y/N] :

Since its intent is to determine whether we build the GRIB2 libraries from source, how about something like:

[GRIB2] Build GRIB2 libraries (zlib, png, JasPer) from source? Default [N] [y/N]:

[mgduda]

As a suggestion for

[WRF] Try to find WRF for metgrid/geogrid?
      Requires WRF path set or ../WRF or ../wrf to exist.
      Valid input paths are WRF_ROOT or WRF_DIR, set via env var or passed into cmake.
      !!! If no valid path is found, these targets will be skipped !!!
      Default [Y] [Y/n] :

what do you think about

[WRF] Attempt to find compiled WRF model (needed for geogrid and metgrid)? Default [Y] [Y/n] :
The following methods will be used to find the compiled WRF model:
    (1) Check for WRF in ../WRF
    (2) Check for WRF in the directory specified by the environment variable WRF_DIR
    (3) Check for WRF in the directory passed as an option to CMake with -DWRF_DIR=<path>

? If --nowrf is specified (in this case, by choosing n) it shouldn't be a failure if WRF is not found. But I think it should result in failure if the user chooses y and WRF is not found.

[islas]

Sorry, in my previous reply I had meant "if --no-wrf is not specified and no potential* WRF is found do not fail" to allow for MPAS builds to not always require an additional arg, well in this case option control.

It sounds like this isn't the way you're leaning as that deviates from the current logic and admittedly would look like silent failure to a normal WRF user. Maybe this mode is something we consider when MPAS becomes more the default.

*edit: Changed to potential so if WRF is somewhere (passed in, env, or ../ dirs exist) it does assume you really DO want WRF and fails. Thus the only false positive where this option would be ignored is if no WRF location was provided by any means.

---

For the prompt, I'm using python input so I don't have a lot of control on text output after the query line but before input is registered.

---

I would assume the order of (1), (2), (3) correlates to precedence. With that in mind (using alphas to avoid confusion with aforementioned #s): a. The search order is slightly reversed, *_ROOT input > env *_ROOT > env *_DIR > implicit dirs like ../WRF b. *_ROOT is a standard and preferred search mechanism for explicit package locations. This can be disabled but I don't recommend that.

Personally, I'd like to avoid duplicating CMake documentation in the prompt and rather reference it. I don't know if this is practically feasible as most users would be accustomed to a prompt with explicit instructions.

[mgduda]

Thanks for the additional explanation. I have a few proposals regarding the detection and use of compiled WRF:

1. There should be only one name for the environment variable and CMake input (i.e., we should not support both WRF_ROOT and WRF_DIR). Since the CMake-based build system already departs from existing behavior, and assuming it's preferable, I think going with WRF_ROOT would be fine.
2. There should be no silent failures. If a user has indicated that they would like to locate compiled WRF code, and compiled WRF code is not found, the build should stop.
3. If there's a preference for not providing details in the prompt for locating compiled WRF code, I would propose having a single-line prompt as with [MPI] and [GRIB2], perhaps

   [WRF] Attempt to find compiled WRF model (needed for geogrid and metgrid)? Default [Y] [Y/n] :

4. If the user indicates that they want to link with WRF (answering y to the prompt), and a compiled WRF model is not found, then we should provide a detailed and informative message like

   Compiled WRF model was not found. Please try of the following:
   (1) provide the option -DWRF_ROOT=<path> to the configure_new script;
   (2) set the environment variable WRF_ROOT to the directory in which the WRF model code was compiled; or
   (3) compile the WRF model code in ../WRF .

[islas]

How about:

CMake Error at CMakeLists.txt:112 (message):
  Compiled WRF model not found.  Please ensure a compiled WRF exists and can
  be found by trying one of the following :

  (1) provide -DWRF_ROOT=<path to install location> to CMake, if done via the
  configure_new script use -- before option or see ./configure_new -h for
  more info

  (2) set the environment variable WRF_ROOT to the path to the install
  location

  (3) compile the WRF model code in ../WRF using the default install location

  More detail on search modes of CMake can be found at

  https://cmake.org/cmake/help/latest/command/find_package.html#search-modes

-- Configuring incomplete, errors occurred!
See also "/home/aislas/wrf-model/wps/_build/CMakeFiles/CMakeOutput.log".

?

This would be the direct output of CMake. Some minor edits to the wording to reference CMake vs ./configure_new mostly to note -- option passing mechanic, and state that the path must point to the install location or use the default install for ../WRF option.

Total output would then look like

./configure_new 
Using default build directory : _build
Using default install directory : /home/aislas/wrf-model/wps/install
0   Linux         gfortran    /    gcc         /    mpif90      /    mpicc      gfortran
!! - Compiler not found, some configurations will not work and will be hidden
Select configuration [0-0] Default [0] (note !!)  : 
[DM]    Use MPI?    Default [N] [y/N] : 
[GRIB2] Build GRIB2 libraries (zlib, libpng, JasPer) from source in external/? Default [N] [y/N] : 
[WRF]   Attempt to find compiled WRF model (needed for geogrid and metgrid)? Default [Y] [Y/n] : 
-- The C compiler identification is GNU 11.4.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is GNU 11.4.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The Fortran compiler identification is GNU 11.4.1
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /usr/bin/gfortran - skipped
-- Checking whether /usr/bin/gfortran supports Fortran 90
-- Checking whether /usr/bin/gfortran supports Fortran 90 - yes
-- Setting project version to 4.4
CMake Error at CMakeLists.txt:81 (message):
  Compiled WRF model not found.  Please ensure a compiled WRF exists and can
  be found by trying one of the following :

  (1) provide -DWRF_ROOT=<path to install location> to CMake, if done via the
  configure_new script use -- before option or see ./configure_new -h for
  more info

  (2) set the environment variable WRF_ROOT to the path to the install
  location

  (3) compile the WRF model code in ../WRF using the default install location

  More detail on search modes of CMake can be found at

  https://cmake.org/cmake/help/latest/command/find_package.html#search-modes

-- Configuring incomplete, errors occurred!
See also "/home/aislas/wrf-model/wps/_build/CMakeFiles/CMakeOutput.log".

[mgduda]

@islas That looks good to me!

[mgduda]

@islas Can you comment on the need for all of the changes in the arch/configure_defaults file? Is the use of separate variables for free- and fixed-format flags necessary for CMake?

[islas]

Mentioned earlier in the PR, most of the edits are to get stanzas consistent. RE free- and fixed- format flags : yes sort of. The cmake build ignores $() and CONFIGURE_ variables expansions and puts them together internally in the options logic. Format is also specifically broken out as that is instead handled by Fortran_FORMAT but there are no cmake-equivalents for byteswap and other flags in F*FLAGS so we still need those fed in