Change model_data to model in COM templates

WalterKolczynski-NOAA commented 2 weeks ago

What new functionality do you need?

By NCO's request, model_data should be just model in COM paths.

What are the requirements for the new functionality?

model_data replaced by model.

Acceptance Criteria

[ ] Output that previous landed in model_data now lands in model
[ ] Output is otherwise unchanged

Suggest a solution (optional)

No response

KateFriedman-NOAA commented 1 week ago

@RussTreadon-NOAA @CoryMartin-NOAA FYI about this request from NCO to change model_data subfolder name to just model. I see the following instances of model_data within the gsi_enkf.fd and gdas.cd subcomponents that should be updated to be consistent. I am taking care of the update on the workflow side.

sorc/gsi_enkf.fd/regression/global_enkf.sh:     $nln $datens/$memchar/model_data/atmos/history/${prefix_ens}.atmf00${fhr}.nc sfg_${global_adate}_fhr0${fhr}_${memchar}
sorc/gsi_enkf.fd/regression/global_enkf.sh:            $nln $datens/$memchar/model_data/atmos/history/${prefix_ens}sfcf00${fhr}.nc sfgsfc_${global_adate}_fhr0${fhr}_${memchar}
sorc/gsi_enkf.fd/regression/global_enkf.sh:    $nln $datens/ensstat/model_data/atmos/history/${prefix_ens}.atmf00${fhr}.ensmean.nc sfg_${global_adate}_fhr0${fhr}_ensmean
sorc/gsi_enkf.fd/regression/global_4denvar.sh:dathis=$COMROOTgfs/$dumpges.$PDYg/${cycg}/model_data/atmos/history
sorc/gsi_enkf.fd/regression/global_4denvar.sh:$nln $datens/ensstat/model_data/atmos/history/${prefix_ens}.sfcf006.ensmean.nc         ./sfcf06_anlgrid
sorc/gsi_enkf.fd/regression/global_4denvar.sh:  $nln $datens/$member/model_data/atmos/history/$sigens ${ENS_PATH}sigf${fh}_ens_${member}

sorc/gdas.cd/ush/soca/run_jjobs.py:        model_data = os.path.join(self.com_src, f"{self.RUN}.{self.gPDY}", self.gcyc, "model_data")
sorc/gdas.cd/ush/soca/run_jjobs.py:        com_ocean_history_src = os.path.join(model_data, 'ocean', 'history')
sorc/gdas.cd/ush/soca/run_jjobs.py:        com_ice_history_src = os.path.join(model_data, 'ice', 'history')
sorc/gdas.cd/ush/soca/run_jjobs.py:        com_ice_restart_src = os.path.join(model_data, 'ice', 'restart')
sorc/gdas.cd/ush/soca/marine_recenter.py:                                       'model_data',
sorc/gdas.cd/scripts/exgdas_global_marine_analysis_prep.py:                                  'model_data', domain, 'history')
sorc/gdas.cd/test/atm/global-workflow/jjob_ens_init.sh:    source=$GDASAPP_TESTDATA/lowres/$dpath/$memchar/model_data/atmos
sorc/gdas.cd/test/atm/global-workflow/jjob_ens_init.sh:    source=$GDASAPP_TESTDATA/lowres/$dpath/$memchar/model_data/atmos/history
sorc/gdas.cd/test/atm/global-workflow/jjob_var_init.sh:dpath=gdas.$gPDY/$gcyc/model_data/atmos/history
sorc/gdas.cd/test/atm/global-workflow/jjob_var_init.sh:dpath=gdas.$gPDY/$gcyc/model_data/atmos
sorc/gdas.cd/test/atm/global-workflow/jjob_var_init.sh:    source=$GDASAPP_TESTDATA/lowres/$dpath/$memchar/model_data/atmos
sorc/gdas.cd/test/atm/global-workflow/jjob_var_init.sh:    source=$GDASAPP_TESTDATA/lowres/$dpath/$memchar/model_data/atmos/history
sorc/gdas.cd/test/snow/apply_jedi_incr.sh:RSTDIR=$GDASAPP_TESTDATA/lowres/gdas.$GYMD/$GHR/model_data/atmos/restart
sorc/gdas.cd/test/snow/create_bkg_ens.sh:RSTDIR=$GDASAPP_TESTDATA/lowres/gdas.$GYMD/$GHR/model_data/atmos/restart
sorc/gdas.cd/test/snow/test_imsproc.sh:RSTDIR=$GDASAPP_TESTDATA/lowres/gdas.$GYMD/$GHR/model_data/atmos/restart
sorc/gdas.cd/test/snow/letkfoi_snowda.sh:RSTDIR=$GDASAPP_TESTDATA/lowres/gdas.$GYMD/$GHR/model_data/atmos/restart
sorc/gdas.cd/test/aero/global-workflow/jjob_var_init.sh:dpath=gdas.$gPDY/$gcyc/model_data/atmos
sorc/gdas.cd/test/soca/gw/prep.sh:mkdir -p ${COM}/06/model_data/ice/history
sorc/gdas.cd/test/soca/gw/prep.sh:mkdir -p ${COM}/06/model_data/ice/restart
sorc/gdas.cd/test/soca/gw/prep.sh:mkdir -p ${COM}/06/model_data/ocean/history
sorc/gdas.cd/test/soca/gw/prep.sh:mkdir -p ${COM}/06/model_data/atmos/analysis
sorc/gdas.cd/test/soca/gw/prep.sh:cp ${ice_rst} ${COM}/06/model_data/ice/restart/20180415.120000.cice_model.res.nc
sorc/gdas.cd/test/soca/gw/prep.sh:  cp $ocnf ${COM}/06/model_data/ocean/history/gdas.ocean.t06z.inst.f00$i.nc
sorc/gdas.cd/test/soca/gw/prep.sh:  cp $hist_icef ${COM}/06/model_data/ice/history/gdas.ice.t06z.inst.f00$i.nc
sorc/gdas.cd/test/soca/gw/prep.sh:    cp ${COM}/06/model_data/ocean/history/gdas.ocean.t06z.inst.f003.nc \
sorc/gdas.cd/test/soca/gw/prep.sh:    list_of_ocn_fcst=$(ls ${COM}/06/model_data/${domain}/history/gdas.${domain}.t06z.inst.f*.nc)
sorc/gdas.cd/test/soca/gw/prep.sh:    oceandir=${COMENS}/06/mem00${mem}/model_data/ocean/history
sorc/gdas.cd/test/soca/gw/prep.sh:    icedir=${COMENS}/06/mem00${mem}/model_data/ice/history

KateFriedman-NOAA commented 1 week ago

@GeorgeGayno-NOAA FYI about this request from NCO to change the model_data subfolder name to just model. I see the following instance of model_data within the UFS_UTILS subcomponent. Please update to be consistent, thanks!

sorc/ufs_utils.fd/util/gdas_init/copy_coldstart_files.sh:  SAVEDIR_MODEL_DATA=$SUBDIR/model_data/atmos/input

RussTreadon-NOAA commented 1 week ago

@KateFriedman-NOAA , thank you for letting us know about this NCO requirement. As you note, the g-w changes need to be coordinated with GSI and GDASApp changes. Would you please open a GSI issue and a GDASApp issue to document the required change?

KateFriedman-NOAA commented 1 week ago

@RussTreadon-NOAA As requested, issues opened: https://github.com/NOAA-EMC/GSI/issues/761 & https://github.com/NOAA-EMC/GDASApp/issues/1198

GeorgeGayno-NOAA commented 1 week ago

@GeorgeGayno-NOAA FYI about this request from NCO to change the model_data subfolder name to just model. I see the following instance of model_data within the UFS_UTILS subcomponent. Please update to be consistent, thanks!
sorc/ufs_utils.fd/util/gdas_init/copy_coldstart_files.sh:  SAVEDIR_MODEL_DATA=$SUBDIR/model_data/atmos/input

Will work this under: https://github.com/ufs-community/UFS_UTILS/issues/967

RussTreadon-NOAA commented 1 week ago

NOAA-EMC/GSI has been updated to use model instead of model_data. Done at 6a87460. See GSI PR #762 for details.

RussTreadon-NOAA commented 1 week ago

NOAA-EMC/GDASApp:feature/model has been updated to use model instead of model_data. Build and run GDASApp ctests on Orion inside working copy of PR #2700 g-w. All 48 tests pass but notice that model_data remains. This makes sense sinc PR #2700 g-w still uses model_data. See GDASApp issue #1198 for additional information.

Do we have a g-w branch with all occurrences of model_data replaced with model? This is the branch in which we should test GDASApp branch feature/model.

KateFriedman-NOAA commented 6 days ago

Do we have a g-w branch with all occurrences of model_data replaced with model? This is the branch in which we should test GDASApp branch feature/model.

Hmmmmm, I am considering options for this, one being to merge this work with my issue #2475 work (since I need this work for that issue's work and merging the efforts would reduce work for both of them).

Question for @RussTreadon-NOAA and @GeorgeGayno-NOAA ...are the model_data -> model changes going into your respective develop branches ahead of Orion Rocky9 updates?

RussTreadon-NOAA commented 6 days ago

@KateFriedman-NOAA ,

GDASApp

Orion Rocky 9 is already in GDASApp develop. Done via PR #1180
model_data -> model change remains feature/model. PR not opened since change needs to be coordinated with updates to GDASApp test case data directories on WCOSS2, Hera, and Orion/Hercules

GSI

Orion Rocky 9 on pause (see GSI PR #764) given the increased gsi.x and enkf.x wall times
model_data -> model change is already in GSI develop. Done via PR #762

KateFriedman-NOAA commented 6 days ago

@RussTreadon-NOAA Ok, given the mixture of order between the model_data -> model changes and Orion Rocky9 updates between the subcomponents I will keep this work separate from the issue #2475 work. I will need to create symlinks within the current IC structure to support the model_data -> model changes going in before my issue #2475 work (which also handles that folder name change but which I don't want to mix with the Orion Rocky9 updates). I'll create the symlinks and create a g-w branch with the workflow side changes. Stay tuned. Thanks!

GeorgeGayno-NOAA commented 6 days ago

Do we have a g-w branch with all occurrences of model_data replaced with model? This is the branch in which we should test GDASApp branch feature/model.

Hmmmmm, I am considering options for this, one being to merge this work with my issue #2475 work (since I need this work for that issue's work and merging the efforts would reduce work for both of them).

Question for @RussTreadon-NOAA and @GeorgeGayno-NOAA ...are the model_data -> model changes going into your respective develop branches ahead of Orion Rocky9 updates?

The directory changes were merged yesterday: https://github.com/ufs-community/UFS_UTILS/pull/968

The Orion changes were merged this morning: https://github.com/ufs-community/UFS_UTILS/pull/966

KateFriedman-NOAA commented 6 days ago

Noted, thanks @GeorgeGayno-NOAA !

NOAA-EMC / global-workflow