update WAM RT compatible with v17

WChen-NOAA commented 4 weeks ago

Commit Queue Requirements:

[x] Fill out all sections of this template.
[ ] All sub component pull requests have been reviewed by their code managers.
[ ] Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
[ ] Commit 'test_changes.list' from previous step

Description:

This PR fix issue https://github.com/ufs-community/ufs-weather-model/issues/2282 (related to https://github.com/ufs-community/ufs-weather-model/pull/2173) . The changes are to allow compatibility for WAM regression tests with GFS_v17_p8. Changes have been made to the regression tests for this update. wam_v17.nml.IN is based on the new global template namelist for the RTs. Variable values have been set to fit for WAM RTs. diag_table_wam and field_table_wam_noaer are updated accordingly. The warm-start RTs are replaced with cold-start RTs and the ICs are replaced with cold start ICs.

Commit Message:
UFSWM - update WAM RT compatible with v17

Priority:

Normal

Git Tracking

UFSWM:

Closes #2282

Sub component Pull Requests:

None

UFSWM Blocking Dependencies:

Blocked by #2173

Changes

Regression Test Changes (Please commit test_changes.list):

PR Updates/Changes Baselines. control_wam control_wam_debug

Input data Changes:

Updated input data. FV3_input_data_L149 on HERA soft links are made for RTs, use export INPUTDATA_ROOT=/scratch1/NCEPDEV/stmp2/Wen.Chen/FV3WAM before running tests
Library Changes/Upgrades:

Testing Log:
- RDHPCS
  - [ ] Hera
  - [ ] Orion
  - [ ] Hercules
  - [ ] Jet
  - [ ] Gaea
  - [ ] Derecho
- WCOSS2
  - [ ] Dogwood/Cactus
  - [ ] Acorn
- [ ] CI
- [ ] opnReqTest (complete task if unnecessary)

BrianCurtis-NOAA commented 4 weeks ago

@DusanJovic-NOAA do we have "rules" for the INPUTDATA_ROOT dir for placing things? I would think maybe the WAM items could go into a new directory named WAM_input_data instead of FV3_input_data_L149/FV3_input_data_wam but I don't recall if we had to keep things a certain way.

DusanJovic-NOAA commented 4 weeks ago

@DusanJovic-NOAA do we have "rules" for the INPUTDATA_ROOT dir for placing things? I would think maybe the WAM items could go into a new directory named WAM_input_data instead of FV3_input_data_L149/FV3_input_data_wam but I don't recall if we had to keep things a certain way.

Where are 'wam' input data currently stored? Isn't 'wam' just another configuration of global, uncoupled fv3 configuration. What makes 'wam' a wam compared to control_p8 for example? How are input/IC data used in 'wam' different than what's currently used in other fv3 configurations?

DeniseWorthen commented 4 weeks ago

@BrianCurtis-NOAA Do we know this new test passes all the oRTs?

BrianCurtis-NOAA commented 4 weeks ago

@BrianCurtis-NOAA Do we know this new test passes all the oRTs?

This doesn't seem to be a new test, but it won't hurt to have @zach1221 run the ORT on it once all requested changes are taken care of.

DeniseWorthen commented 3 weeks ago

@BrianCurtis-NOAA Do we know this new test passes all the oRTs?

This doesn't seem to be a new test, but it won't hurt to have @zach1221 run the ORT on it once all requested changes are taken care of.

Thanks @zach1221. It appears this test is different enough (v16->v17) that it warrants treatment as "new".

zach1221 commented 3 weeks ago

Hi, @WChen-NOAA can you please sync up this PR's branch so I can begin ORTs?

WChen-NOAA commented 3 weeks ago

@zach1221 This is the first time I am working on the UFS-WM PRs. Confused by "sync up this PR's branch so I can begin ORTs". Is this to update my branch with the UFS-WM develop branch?

BrianCurtis-NOAA commented 3 weeks ago

@zach1221 This is the first time I am working on the UFS-WM PRs. Confused by "sync up this PR's branch so I can begin ORTs". Is this to update my branch with the UFS-WM develop branch?

You are correct.

WChen-NOAA commented 3 weeks ago

the ICs needs to use the cold ICs in (HERA) /scratch1/NCEPDEV/global/Wen.Chen/models/PRs/FV3_input_data_L149

BrianCurtis-NOAA commented 3 weeks ago

the ICs needs to use the cold ICs in (HERA) /scratch1/NCEPDEV/global/Wen.Chen/models/PRs/FV3_input_data_L149

Are there overwrites of files in FV3_input_data_L149 or all new files? This would help us determine if we need to create a new input-data directory for these changes.

WChen-NOAA commented 3 weeks ago

@BrianCurtis-NOAA Please overwrite the files in FV3_input_data_L149.

WChen-NOAA commented 3 weeks ago

could I start the full suite RTs on HERA? or waiting for a while?

BrianCurtis-NOAA commented 3 weeks ago

@BrianCurtis-NOAA Please overwrite the files in FV3_input_data_L149.

I think we should name it FV3_input_data_L149_wam on the input-data directory so that we don't need to create a new input-data directory for this. Could you make changes in your code to accommodate this?

Only exception would be knowing that nobody else uses L149 and nobody in the WAM group would need the older inputs in the future. I'd rather not overwrite, though.

BrianCurtis-NOAA commented 3 weeks ago

could I start the full suite RTs on HERA? or waiting for a while?

You can run these at any time your code is ready. We expect that no changes (minus keeping up to date with develop branch) would be needed afterwards.

zach1221 commented 3 weeks ago

@WChen-NOAA I assume directory INPUT_C should be created as new in FV3_input_data_L149, as it doesn't currently exist, or should the files in /scratch1/NCEPDEV/stmp2/Wen.Chen/FV3WAM/fv3wam_data_src/INPUT_C be copied to /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20240501/FV3_input_data_L149/INPUT ?

WChen-NOAA commented 3 weeks ago

@BrianCurtis-NOAA Please overwrite the files in FV3_input_data_L149.

I think we should name it FV3_input_data_L149_wam on the input-data directory so that we don't need to create a new input-data directory for this. Could you make changes in your code to accommodate this?

Only exception would be knowing that nobody else uses L149 and nobody in the WAM group would need the older inputs in the future. I'd rather not overwrite, though.

@zach1221 I am doing with the code as Brian suggested. Could be better for future works.

WChen-NOAA commented 3 weeks ago

@BrianCurtis-NOAA @zach1221 The inputdir was made to FV3_input_data_L149_wam (sorry, still did not figure out how to properly quote the code changes), The wam RTs still work. HERA location of ICs: /scratch1/NCEPDEV/global/Wen.Chen/models/PRs/FV3_input_data_L149_wam

WChen-NOAA commented 2 weeks ago

@BrianCurtis-NOAA When I running the full suite RTs. I found:

Only the head of stochastic_physics is different. The head of stochastic_physics is "fad2fe" the same as the current hash appeared on UFS-WM develop branch but it is different with the hash in HERA RT log (ecdff7).
I have skipped tests. e.g. "SKIPPED (TEST DOES NOT GENERATE BASELINE)....cpld_restart_p8_intel"
The failed wam RTs (New baseline data are needed since the ICs and configurations are changed. I use the -c option for all suite RTs or I make soft links of baseline data for other RTs but use my new baseline for wam RTs?)
test_changes.list file is generated and only contains wam RTs.

FernandoAndrade-NOAA commented 2 weeks ago

Data synced to Jet. I will sync Gaea once it's available again after maintenance. @jkbk2004 FYI

zach1221 commented 2 weeks ago

FV3_input_data_L149_wam data synced to hercules, orion and derecho.

zach1221 commented 2 weeks ago

@BrianCurtis-NOAA When I running the full suite RTs. I found:

Only the head of stochastic_physics is different. The head of stochastic_physics is "fad2fe" the same as the current hash appeared on UFS-WM develop branch but it is different with the hash in HERA RT log (ecdff7).

I have skipped tests. e.g. "SKIPPED (TEST DOES NOT GENERATE BASELINE)....cpld_restart_p8_intel"

The failed wam RTs (New baseline data are needed since the ICs and configurations are changed. I use the -c option for all suite RTs or I make soft links of baseline data for other RTs but use my new baseline for wam RTs?)

test_changes.list file is generated and only contains wam RTs.

@WChen-NOAA are you able to push your updated test_changes.list file to this PR, and then sync up the branch?

FernandoAndrade-NOAA commented 2 weeks ago

Input data synced to Gaea.

WChen-NOAA commented 6 days ago

@WChen-NOAA are you able to push your updated test_changes.list file to this PR, and then sync up the branch?

@zach1221 @BrianCurtis-NOAA The mentioned steps have been done.

jkbk2004 commented 18 hours ago

@WChen-NOAA Can you continue to sync up the branch? It might be possible to commit this pr this week.

WChen-NOAA commented 4 hours ago

@jkbk2004 Done

ufs-community / ufs-weather-model