Need plan for upcoming releases

ligiabernardet commented 4 years ago

We need to create a plan for upcoming releases of the MR W App, both for fixes 1.0.x and for upgrades, 1.x. @arunchawla-NOAA @mvertens @lbnance @junwang-noaa @ceceliadid

Must decide which branches will be used for the release (some offline emails indicate it will be the same release branches already used for the 1.0.0 release because the main development branches have too many new capabilities).

For upcoming releases, @llpcarson and I would like to point most of the documentation URLs to the head of release branch (not the tag). This will facilitate adding bug fixes and clarifications to the documentation as we get feedback from the community.

So far I have identified the following list of potential upgrades to be included:

CCPP Framework: now works with Python 3 (therefore the whole App can be said to work with Python 3 - need testing); PR for ufs-weather-model: https://github.com/ufs-community/ufs-weather-model/pull/92
ufs-utils/chgres: Patch to account for bad temperature data
ufs-mrweather-app and ufs-weather-model/parm/ccpp_v16beta_c96.nml.IN: change diffusion according to https://forums.ufscommunity.org/threads/strength-second-order-damping-top-layers
NCEPLIBS-external: Documentation updates have gone into master with https://github.com/NOAA-EMC/NCEPLIBS-external/pull/43
ufs-mrweather-app Documentation: Suggestion from Neil via @climbfuji: I think in Section 2.1 (quick start) there needs to be a note and link to section 5.2 regarding what to add/do when trying this on a supported-but-not-preconfigured platform.
CCPP Physics: minor updates to Scientific Documentation

ligiabernardet commented 4 years ago

@jedwards4b

jedwards4b commented 4 years ago

@ligiabernardet Thanks!

arunchawla-NOAA commented 4 years ago

@ligiabernardet there is also the update that needs to come to chgres_cube to change from using nemsio files to nc files. These files will replace the nemsio files when gfsv16 goes into operations. Because of the amount of testing involved I want to be careful about the number of upgrades we do

ligiabernardet commented 4 years ago

I agree that we need to be careful about the number of upgrades we do because of limited staffing. Mostly we need to create a plan. If we have agreement that the innovations for 1.1 should go to existing public release branches, that is a good start. We can issue this issue to collect a list of the desired upgrades. Updated list below.

CCPP Framework: now works with Python 3 (therefore the whole App can be said to work with Python 3 - need testing); PR for ufs-weather-model: ufs-community/ufs-weather-model#92
ufs-utils/chgres: Patch to account for bad temperature data and upgrade to ingest NetCDF files (iin addition to NEMSIO and GRIB2 files) in preparation for GFSv16 operational model (it will output NetCDF)
ufs-mrweather-app and ufs-weather-model/parm/ccpp_v16beta_c96.nml.IN: change diffusion according to https://forums.ufscommunity.org/threads/strength-second-order-damping-top-layers
NCEPLIBS-external: Documentation updates have gone into master with NOAA-EMC/NCEPLIBS-external#43
ufs-mrweather-app Documentation: Suggestion from Neil via @climbfuji: I think in Section 2.1 (quick start) there needs to be a note and link to section 5.2 regarding what to add/do when trying this on a supported-but-not-preconfigured platform.
CCPP Physics: minor updates to Scientific Documentation

ligiabernardet commented 4 years ago

Updated list by request from @climbfuji.

CCPP Framework: now works with Python 3 (therefore the whole App can be said to work with Python 3 - need testing); PR for ufs-weather-model: ufs-community/ufs-weather-model#92; also documentation updates (https://github.com/NCAR/ccpp-framework/pull/274)
ufs-utils/chgres: Patch to account for bad temperature data and upgrade to ingest NetCDF files (iin addition to NEMSIO and GRIB2 files) in preparation for GFSv16 operational model (it will output NetCDF)
ufs-mrweather-app and ufs-weather-model/parm/ccpp_v16beta_c96.nml.IN: change diffusion according to https://forums.ufscommunity.org/threads/strength-second-order-damping-top-layers
NCEPLIBS-external: Documentation updates have gone into master with NOAA-EMC/NCEPLIBS-external#43
ufs-mrweather-app Documentation: Suggestion from Neil via @climbfuji: I think in Section 2.1 (quick start) there needs to be a note and link to section 5.2 regarding what to add/do when trying this on a supported-but-not-preconfigured platform.
CCPP Physics: minor updates to Scientific Documentation

ligiabernardet commented 4 years ago

@climbfuji You mentioned we should consider an upgrade to the build system. Would you be able to explain what that entails, and which code repositories that would affect?

mvertens commented 4 years ago

It would be helpful to also describe how these upgrades would interact with the CIME build system in the release.

On Mon, Apr 20, 2020 at 11:43 AM ligiabernardet notifications@github.com wrote:

@climbfuji https://github.com/climbfuji You mentioned we should consider an upgrade to the build system. Would you be able to explain what that entails, and which code repositories that would affect?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/149#issuecomment-616708915, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4XCEYOYW45LRTY6UZVOYLRNSCS3ANCNFSM4LUJLWYQ .

-- Mariana Vertenstein CESM Software Engineering Group Head National Center for Atmospheric Research Boulder, Colorado Office 303-497-1349 Email: mvertens@ucar.edu

ligiabernardet commented 4 years ago

Updated list of potential updates

CCPP Framework: now works with Python 3 (therefore the whole App can be said to work with Python 3 - need testing); PR for ufs-weather-model: ufs-community/ufs-weather-model#92; also documentation updates (NCAR/ccpp-framework#274)
ufs-utils/chgres: Patch to account for bad temperature data and upgrade to ingest NetCDF files (iin addition to NEMSIO and GRIB2 files) in preparation for GFSv16 operational model (it will output NetCDF)
ufs-mrweather-app and ufs-weather-model/parm/ccpp_v16beta_c96.nml.IN: change diffusion according to https://forums.ufscommunity.org/threads/strength-second-order-damping-top-layers
NCEPLIBS-external: Documentation updates have gone into master with NOAA-EMC/NCEPLIBS-external#43
ufs-mrweather-app Documentation: Suggestion from Neil via @climbfuji: I think in Section 2.1 (quick start) there needs to be a note and link to section 5.2 regarding what to add/do when trying this on a supported-but-not-preconfigured platform.
CCPP Physics: minor updates to Scientific Documentation
Build System upgrades: NCEPLIBS-external, NCEPLIBS (add missing parts for SRW release); these build system updates should be mostly transparent to the CIME build system for the release
Incorporate feedback from GST
bugfixes for no_sst suites (see https://github.com/NOAA-EMC/fv3atm/pull/110 and https://github.com/ufs-community/ufs-weather-model/pull/118)

ligiabernardet commented 4 years ago

App doc: Add info about how one can see the entire namelist before they make a change.
Documentation displays (ReadtheDocs etc.) for v1.0 and new releases will not point to tags but instead to head of release branches

arunchawla-NOAA commented 4 years ago

Getting initial condition files. The nemsio files will not be available once the gfs v16 goes into operations and will be replaced by NetCDF files. Need to determine how CIME will handle this change. Also NCDC website has been giving problems

arunchawla-NOAA commented 4 years ago

See if we can add this

https://github.com/NOAA-EMC/UFS_UTILS/issues/97

ligiabernardet commented 4 years ago

Add metadata to chgres output https://github.com/NOAA-EMC/UFS_UTILS/issues/97

fossell commented 4 years ago

UPP had bug fix merged into branch v1.0, details here in EMC_post Issue 129 . Should confirm this was tested and/or test again. @aerorahul contributed this bug fix.

climbfuji commented 4 years ago

Also: need update for jet (filesystem changes)

climbfuji commented 4 years ago

I started collecting changes to the ufs-weather-model and submodules, see https://github.com/ufs-community/ufs-weather-model/pull/149 and associated PRs (currently in draft status).

fossell commented 4 years ago

Is there a time line for when component repos (e.g. UPP) should be creating v1.1 branch (from public-v1 branch as discussed)? Should this be created now so testing can use that v1.1 branch and any necessary fixes go into that new branch? Or wait until closer to release, and while waiting any new features/fixes go into branch public-v1?

climbfuji commented 4 years ago

Is there a time line for when component repos (e.g. UPP) should be creating v1.1 branch (from public-v1 branch as discussed)? Should this be created now so testing can use that v1.1 branch and any necessary fixes go into that new branch? Or wait until closer to release, and while waiting any new features/fixes go into branch public-v1?

Bad time to ask this question. I wasted hours today figuring this out. See https://github.com/ufs-community/ufs-weather-model/pull/149.

fossell commented 4 years ago

@climbfuji Yikes! Ok, I'll wait until this is hashed out. No rush on UPP side, just wanted to stay informed of process.

ligiabernardet commented 4 years ago

I am closing this issue since the specific work needed for the 1.1.0 release has been distributed through various issues.

ufs-community / ufs-mrweather-app

Need plan for upcoming releases #149