search

ufs-community / ufs-mrweather-app

UFS Medium-Range Weather Application
Other
23 stars 23 forks source link

Build system improvements for release v1.1 #157

Closed climbfuji closed 4 years ago

climbfuji commented 4 years ago

We need to update the NCEPLIBS-external build system to address some issues users have reported when building netCDF or using existing netCDF installations. Possibly/optionally also update NCEPLIBS itself, although the latter may not be required (if the NCEPLIBS-external update is done correctly).

ligiabernardet commented 4 years ago

Do you know if documentation updates will be needed to accompany this change?

climbfuji commented 4 years ago

Do you know if documentation updates will be needed to accompany this change?

Not entirely sure. Ideally, no - changes are under the hood only. And as far as I know, you don't include machine-specific instructions (which are in NCEPLIBS-externa/doc) or other information that is in NCEPLIBS-external README.md and/or NCEPLIBS README.md, or?

ligiabernardet commented 4 years ago

If updates are needed, can you and the build team update README.md and the machine-specific files in NCEPLIBS-external/doc? Then let me know so we can review it.

climbfuji commented 4 years ago

Sure, this will be part of the development and testing.

On Jun 26, 2020, at 12:05 PM, ligiabernardet notifications@github.com wrote:

If updates are needed, can you and the build team update README.md and the machine-specific files in NCEPLIBS-external/doc? Then let me know so we can review it.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/157#issuecomment-650319466, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5C2RO4I5AV6VO2DWHMWDTRYTPOLANCNFSM4OJOLOCQ.

ligiabernardet commented 4 years ago

@climbfuji Do you have a timeline for this build update to be complete?

climbfuji commented 4 years ago

@climbfuji Do you have a timeline for this build update to be complete?

We are making good progress, yes!

ligiabernardet commented 4 years ago

@climbfuji Can you please provide an update?

climbfuji commented 4 years ago

@climbfuji Can you please provide an update?

Here is the link to the ongoing discussion: https://github.com/ufs-community/ufs-mrweather-app/issues/154

climbfuji commented 4 years ago

Someone could test these new libraries on here with the netCDF data. How to load the modules and compile the ufs-weather-model (branch release/public-v1):

module purge
module load intel/18.0.5.274
module load impi/2018.0.4
module load netcdf/4.7.0
module load cmake/3.16.1

export CC=icc
export CXX=icpc
export FC=ifort

module use -a /scratch1/BMC/gmtb/software/modulefiles/intel-18.0.5.274/impi-2018.0.4
module load NCEPlibs/1.1.0beta01

export CMAKE_Platform=hera.intel
./build.sh 2>&1 | tee build.log

Or the CIME config needs to be updated manually to load these NCEPLIBS instead of the default NCEPLIBS/1.0.0.

Please let me know if someone wants to test this installation. Thanks!

ligiabernardet commented 4 years ago

When trying to build the branch release/public-v1 of the ufs-weather-model on Hera, I am getting a failure in the ccpp_prebuild. ccpp_prebuild.py: error: unrecognized arguments: --static Does the build.sh in the branch release/public-v1 of the ufs-weather-model need to be updated?

climbfuji commented 4 years ago

When trying to build the branch release/public-v1 of the ufs-weather-model on Hera, I am getting a failure in the ccpp_prebuild. ccpp_prebuild.py: error: unrecognized arguments: --static Does the build.sh in the branch release/public-v1 of the ufs-weather-model need to be updated?

I tried the following successfully on hera:

git clone -b release/public-v1 --recursive https://github.com/ufs-community/ufs-weather-model
cd ufs-weather-model
module purge
module load intel/18.0.5.274
module load impi/2018.0.4
module load netcdf/4.7.0
module load cmake/3.16.1
export CC=icc
export CXX=icpc
export FC=ifort
module use -a /scratch1/BMC/gmtb/software/modulefiles/intel-18.0.5.274/impi-2018.0.4
module load NCEPlibs/1.1.0beta01
export CMAKE_Platform=hera.intel
./build.sh 2>&1 | tee build.log

Is it possible that you checked out the default branches of ufs-weather-model and switched to release/public-v1 in the top-level repo only, but not in the submodules?

ligiabernardet commented 4 years ago

Yes, that is what I did. I will checkout with the -b release/public-v1 and make sure I am using release/public-v1 for all components.

ligiabernardet commented 4 years ago

I was able to build the ufs-weather-model on Hera with the new libraries. No surprise since I am basically repeating the steps performed by @climbfuji. But it was good to confirm.

climbfuji commented 4 years ago

NCEPLIBS-external and its submodules are ready, in particular the ones we control. Tags ufs-v1.1.0 have been created for:

https://github.com/NOAA-EMC/CMakeModules https://github.com/NOAA-EMC/NCEPLIBS-wgrib2 https://github.com/NOAA-EMC/NCEPLIBS-external

Note that NCEPLIBS-external will be retagged with documentation-only updates (README.md and doc/README_*.txt) after the libraries have been installed and tested everywhere.

For NCEPLIBS, I am waiting for the approval of a PR for UFS_UTILS, https://github.com/NOAA-EMC/UFS_UTILS/pull/131, and a following PR for NCEPLIBS itself, https://github.com/NOAA-EMC/NCEPLIBS/pull/102. Once these are merged, we are ready to create tags ufs-v1.1.0 for the NCEPLIBS-* submodules and the NCEPLIBS umbrella repository.

What is the current status of testing the netCDF ingest and running the model with the new chgres_cube.exe? Can I help?

ligiabernardet commented 4 years ago

I am hoping to run chgres_cube standalone today with GRIB2 and netCDF on Hera. The input datasets are staged on Hera if others want to try it.

/scratch1/BMC/gmtb/Ligia.Bernardet/myUFS_INPUT/ufs_inputdata/icfiles/202002/20200202/

On Wed, Aug 12, 2020 at 9:07 AM Dom Heinzeller notifications@github.com wrote:

NCEPLIBS-external and its submodules are ready, in particular the ones we control. Tags ufs-v1.1.0 have been created for:

https://github.com/NOAA-EMC/CMakeModules https://github.com/NOAA-EMC/NCEPLIBS-wgrib2 https://github.com/NOAA-EMC/NCEPLIBS-external

Note that NCEPLIBS-external will be retagged with documentation-only updates (README.md and doc/README_*.txt) after the libraries have been installed and tested everywhere.

For NCEPLIBS, I am waiting for the approval of a PR for UFS_UTILS, NOAA-EMC/UFS_UTILS#131 https://github.com/NOAA-EMC/UFS_UTILS/pull/131, and a following PR for NCEPLIBS itself, NOAA-EMC/NCEPLIBS#102 https://github.com/NOAA-EMC/NCEPLIBS/pull/102. Once these are merged, we are ready to create tags ufs-v1.1.0 for the NCEPLIBS-* submodules and the NCEPLIBS umbrella repository.

What is the current status of testing the netCDF ingest and running the model with the new chgres_cube.exe? Can I help?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/157#issuecomment-672930480, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7WQASPJXE7A26ZWMXPPE3SAKVZ7ANCNFSM4OJOLOCQ .

climbfuji commented 4 years ago

I am hoping to run chgres_cube standalone today with GRIB2 and netCDF on Hera. The input datasets are staged on Hera if others want to try it. /scratch1/BMC/gmtb/Ligia.Bernardet/myUFS_INPUT/ufs_inputdata/icfiles/202002/20200202/ …

Great, please let me know if I can help (and learn how to run chgres_cube.exe standalone). We also need to set up a run directory for testing those initial conditions, I guess.

ligiabernardet commented 4 years ago

I have never run chgres_cube standalone either. But this User's Guide should help! https://ufs-utils.readthedocs.io/en/ufs-v1.0.0/

On Wed, Aug 12, 2020 at 9:23 AM Dom Heinzeller notifications@github.com wrote:

I am hoping to run chgres_cube standalone today with GRIB2 and netCDF on Hera. The input datasets are staged on Hera if others want to try it. /scratch1/BMC/gmtb/Ligia.Bernardet/myUFS_INPUT/ufsinputdata/icfiles/202002/20200202/ … <#m-8429818165356688788_>

Great, please let me know if I can help (and learn how to run chgres_cube.exe standalone). We also need to set up a run directory for testing those initial conditions, I guess.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/157#issuecomment-672940511, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7WQAWFCOPWRYB275EQVXLSAKXVZANCNFSM4OJOLOCQ .

climbfuji commented 4 years ago

Update on the NCEPLIBS. I rolled out NCEPLIBS v1.1.0 on all preconfigured platforms (level 1) and created the modules, and compiled it successfully on all supported platforms (level 2/3). I also compiled the ufs-weather-model release/public-v1 branch on all of these platforms using build.sh:

hera.intel, cheyenne.intel, cheyenne.gnu, gaea.intel, jet.intel, orion.intel, stampede.intel, macosx.gnu (gcc+gfortran-9 & clang+gfortran-9, gcc+gfortran-10, clang+gfortran-10), Redhat 8 on Amazon (gcc+gfortran-9), Ubuntu 18 (gcc+gfortran-8).

There is an outstanding PR for compiling the ufs-weather-model with gfortran-10 that will need to be merged: https://github.com/ufs-community/ufs-weather-model/pull/186

climbfuji commented 4 years ago

Tags ufs-v1.0.0 have been created for NCEPLIBS-*, NCEPLIBS-external and NCEPLIBS. Rolled out on all level 1 systems and tested on all level-2 and level-3 systems as listed on the updated page https://github.com/ufs-community/ufs/wiki/Supported-Platforms-and-Compilers. I think this issue can be closed.

ligiabernardet commented 4 years ago

This work is complete and ready for further testing.