ufs-community / ufs-weather-model

UFS Weather Model
Other
129 stars 238 forks source link

Add a RT for SFS with hydrostatic options #2255

Closed XiaqiongZhou-NOAA closed 22 hours ago

XiaqiongZhou-NOAA commented 2 weeks ago

Commit Queue Requirements:

Add a RT for the seasonal forecast system with hydrostatic options. Compiling flags -DUSE_COND and -DMOIST_CAPPA are removed if this is a hydrostatic run. inline post is modified to output omega and HGT for hydrostatic runs.

Commit Message:

* UFSWM  - Add a RT for SFS with hydrostatic options
  * FV3 - Remove -DUSE_COND and -DMOIST_CAPPA compiling flags for hydrostatic runs;
              Fix the issue to write out geopotential height for inline post in hydrostatic runs; 
              Use omega from the model if defined in diag_table instead calculated in post
       * atmos_cubed_sphere - Remove -DUSE_COND and -DMOIST_CAPPA compiling flags for hydrostatic runs

Priority:

Sub component Pull Requests:

Changes

Regression Test Changes (Please commit test_changes.list):

Input data Changes:

Library Changes/Upgrades:


Testing Log:

zach1221 commented 6 days ago

@XiaqiongZhou-NOAA running ORTs against the new cpld_gfsv17_sfs case with gnu compilers. Right now getting some failures on the bit, debug, standard, decomp, and restart ORTs. The compile is fine, the failure occurs during the test run. fyi @jkbk2004 @BrianCurtis-NOAA

"An MPI communication peer process has unexpectedly disconnected."

180: *** WAVEWATCH III ERROR IN W3IOGR : 180: ERROR IN READING FROM mod_def.ww3 FILE 180: IOSTAT = 5016 MOD DEF FILE WAS GENERATED WITH A DIFFERENT 180: WW3 VERSION OR USING A DIFFERENT SWITCH FILE. 180: MAKE SURE WW3_GRID IS COMPILED WITH SAME SWITCH 180: AS WW3_SHEL OR WW3_MULTI, RUN WW3_GRID AGAIN 180: AND THEN TRY AGAIN THE PROGRAM YOU JUST USED.

Example test experiment directory: /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_2606214/bit_base_bit_base/ looking at err and out logs.

BrianCurtis-NOAA commented 6 days ago

Check that the mod_def file from WW3 is the correct one

JessicaMeixner-NOAA commented 6 days ago

@BrianCurtis-NOAA is correct. Based on rt.sh and your test, you should be using /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20221101/WW3_input_data_20240214/mod_def.exp.global_270k for your mod_def, but:

Binary files /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_2606214/bit_base_bit_base/mod_def.ww3 and /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20221101/WW3_input_data_20240214/mod_def.exp.global_270k differ

I think you're using the wrong WW3_input data somehow as: /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_2606214/bit_base_bit_base/mod_def.ww3 /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20221101/WW3_input_data_20220624/mod_def.exp.global_270k are the same.

NickSzapiro-NOAA commented 6 days ago

@zach1221 ORT INPUTDATA_ROOT_WW3 needs to match rt.sh: https://github.com/ufs-community/ufs-weather-model/blob/develop/tests/opnReqTest#L636 https://github.com/ufs-community/ufs-weather-model/blob/develop/tests/rt.sh#L1037

zach1221 commented 5 days ago

ORTs passed with WW3_input_data_20240214.

zach1221 commented 3 days ago

@XiaqiongZhou-NOAA @NickSzapiro-NOAA Can you please sync up the branch for this PR?

XiaqiongZhou-NOAA commented 3 days ago

The branch is synced.

On Mon, May 13, 2024 at 1:55 PM zach1221 @.***> wrote:

@XiaqiongZhou-NOAA https://github.com/XiaqiongZhou-NOAA @NickSzapiro-NOAA https://github.com/NickSzapiro-NOAA Can you please sync up the branch for this PR?

— Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-weather-model/pull/2255#issuecomment-2108443199, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALQE7UV5MGNEKEKFYEHHMCLZCD5CFAVCNFSM6AAAAABG22ZRWGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBYGQ2DGMJZHE . You are receiving this because you were mentioned.Message ID: @.***>

BrianCurtis-NOAA commented 3 days ago

Looks like FV3ATM is supposed to be at: https://github.com/XiaqiongZhou-NOAA/fv3atm/commit/fc2630c207f2ac116d9c675081220ac010ba8e9f

This hash was after the indentation change I requested.

XiaqiongZhou-NOAA commented 3 days ago

Pointed to new Fv3atm.

On Mon, May 13, 2024 at 2:11 PM Brian Curtis @.***> wrote:

Looks like FV3ATM is supposed to be at: @.*** https://github.com/XiaqiongZhou-NOAA/fv3atm/commit/fc2630c207f2ac116d9c675081220ac010ba8e9f

This hash was after the indentation change I requested.

— Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-weather-model/pull/2255#issuecomment-2108498635, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALQE7UQNTRKC5G6QUTWIYUDZCD65HAVCNFSM6AAAAABG22ZRWGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBYGQ4TQNRTGU . You are receiving this because you were mentioned.Message ID: @.***>

DeniseWorthen commented 3 days ago

@zach1221 This PR is not ready for the commit Q unless all ORT tests are completed and passed. This test is a possible implementation target and must successfully pass all ORTs at the time of commit.

BrianCurtis-NOAA commented 3 days ago

@zach1221 This PR is not ready for the commit Q unless all ORT tests are completed and passed. This test is a possible implementation target and must successfully pass all ORTs at the time of commit.

He's working on getting the logs for it, and he's using all ORT's.

FernandoAndrade-NOAA commented 3 days ago

@XiaqiongZhou-NOAA Please be sure to fill out the "Commit Message" section of the template, it will be the information included with the final commit when a PR is merged.

XiaqiongZhou-NOAA commented 2 days ago

renamed to "cpld_control_sfs"

On Tue, May 14, 2024 at 9:18 AM Denise Worthen @.***> wrote:

@.**** commented on this pull request.

In tests/rt.conf https://github.com/ufs-community/ufs-weather-model/pull/2255#discussion_r1600029912 :

@@ -31,6 +31,10 @@ RUN | cpld_control_gfsv17_iau | - noaacloud RUN | cpld_restart_gfsv17 | - noaacloud | | cpld_control_gfsv17 RUN | cpld_mpi_gfsv17 | - noaacloud | |

+#SFS +COMPILE | s2swa_32bit_pdlib_sfs | intel | -DAPP=S2SWA -D32BIT=ON -DHYDRO=ON -DCCPP_SUITES=FV3_GFS_v17_coupled_p8_ugwpv1 -DPDLIB=ON | - noaacloud | fv3 | +RUN | cpld_sfs_hydro | - noaacloud | baseline |

The name you've chosen does not really align w/ how we name the RT tests. Yes, the configuration may evolve but this would be the "control" for the "sfs w/ hydrostatic=on".

— Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-weather-model/pull/2255#discussion_r1600029912, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALQE7USV2MKMPUF6G5F44PTZCIFLNAVCNFSM6AAAAABG22ZRWGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDANJVGM2TOMRVGY . You are receiving this because you were mentioned.Message ID: @.***>

XiaqiongZhou-NOAA commented 2 days ago

I finished all regression tests and created a new baseline on Orion: /work/noaa/stmp/xzhou/stmp/xzhou/FV3_RT/REGRESSION_TEST Please let me know what's next.

DeniseWorthen commented 2 days ago

@XiaqiongZhou-NOAA Since this is a new test, all the oRTs must pass at the time we commit this test. Those are the tests for repoducibility,debug, restart, mpi decomp, threading etc. The ORT log (see tests/logs/OpnReqTests_control_p8_hera.log as an example) must be committed to show that the new test passes all those opn requirements. I believe @zach1221 has the ORT log ready?

zach1221 commented 2 days ago

@DeniseWorthen all the ORTs are almost ready. I've been dealing with a persistent disk space issue that has somewhat delayed things, but it shouldn't be much longer.

DeniseWorthen commented 2 days ago

@zach1221 It doesn't look like the dbug test passed?

Test dbg_base FAIL
zach1221 commented 2 days ago

@zach1221 It doesn't look like the dbug test passed?

Test dbg_base FAIL

Yah my first attempt I had a failure, and needed to clear some space and re-run. The second attempt passed and should be there.

BrianCurtis-NOAA commented 1 day ago

The issue with Acorn was incorrectly put as cpld_control_c48 while it was really with cpld_warmstart_c48. I doubled the PE's for I/J to account for an issue with the test using too much memory.

zach1221 commented 1 day ago

Disregard the Orion failure message above.

zach1221 commented 1 day ago

I'll follow up on the cubed-sphere sub-pr and we can begin the merge process.

DusanJovic-NOAA commented 1 day ago

The issue with Acorn was incorrectly put as cpld_control_c48 while it was really with cpld_warmstart_c48. I doubled the PE's for I/J to account for an issue with the test using too much memory.

cpld_warmstart_c48 should need approximately same amount of memory as cpld_control_c48? How did this test work before? Which change in this PR is causing this test to require more memory now?

BrianCurtis-NOAA commented 1 day ago

The issue with Acorn was incorrectly put as cpld_control_c48 while it was really with cpld_warmstart_c48. I doubled the PE's for I/J to account for an issue with the test using too much memory.

cpld_warmstart_c48 should need approximately same amount of memory as cpld_control_c48? How did this test work before? Which change in this PR is causing this test to require more memory now?

I had the same questions. Denise mentioned the same to me earlier as well. The run completed fully (as in generated all the sfcf and atmf files it needed) but right at the end it had this, and the comparisons were not run:

  Memuse(MB) at Memory HiWaterMark=  1.429E+02  1.683E+02  1.019E+01  1.508E+02
Application ca116624-ed06-4846-8cb0-deb5a0be1c61 resources: utime=1039s stime=1207s maxrss=2697772KB inblock=3180456 oublock=4777392 minflt=1725554 majflt=453 nvcsw=125660 nivcsw=6252
DusanJovic-NOAA commented 1 day ago

The issue with Acorn was incorrectly put as cpld_control_c48 while it was really with cpld_warmstart_c48. I doubled the PE's for I/J to account for an issue with the test using too much memory.

cpld_warmstart_c48 should need approximately same amount of memory as cpld_control_c48? How did this test work before? Which change in this PR is causing this test to require more memory now?

I had the same questions. Denise mentioned the same to me earlier as well. The run completed fully (as in generated all the sfcf and atmf files it needed) but right at the end it had this, and the comparisons were not run:

  Memuse(MB) at Memory HiWaterMark=  1.429E+02  1.683E+02  1.019E+01  1.508E+02
Application ca116624-ed06-4846-8cb0-deb5a0be1c61 resources: utime=1039s stime=1207s maxrss=2697772KB inblock=3180456 oublock=4777392 minflt=1725554 majflt=453 nvcsw=125660 nivcsw=6252

This does not indicate any error.

If the model finished successfully then the issue is not that the test is using too much memory. There must be some other reason why the comparisons were not run. Please revert to original setup and try again.

BrianCurtis-NOAA commented 1 day ago
====START OF ACORN REGRESSION TESTING LOG====

UFSWM hash used in testing:
16f3271ba7e5266662904d1524d87a80f97bf9bf

Submodule hashes used in testing:
 37cbb7d6840ae7515a9a8f0dfd4d89461b3396d1 AQM (v0.2.0-37-g37cbb7d)
 be5d28fd1b60522e6fc98aefeead20e6aac3530b AQM/src/model/CMAQ (CMAQv5.2.1_07Feb2018-198-gbe5d28fd1)
 3d7067a523b8557058755289e4275f5f5c985daf CDEPS-interface/CDEPS (cdeps0.4.17-40-g3d7067a)
 9c0cd8d3ba5963d01711094a3c81b6358fcb8367 CICE-interface/CICE (CICE6.0.0-448-g9c0cd8d)
 f6ff8f7c4d4cb6feabe3651b13204cf43fc948e3 CICE-interface/CICE/icepack (Icepack1.1.0-182-gf6ff8f7)
 4e19850cb083bc474b7cde5dc2f8506ec74cc442 CMEPS-interface/CMEPS (cmeps_v0.4.1-2306-g4e19850)
 cabd7753ae17f7bfcc6dad56daf10868aa51c3f4 CMakeModules (v1.0.0-28-gcabd775)
 b329c2ccc4c5c6936cb1f548eef83d2a1ee36ae2 FV3 (remotes/origin/hydro)
 aabfdccae440da7389195762601db7fcc09f6f11 FV3/atmos_cubed_sphere (remotes/origin/hydro)
 741212e4962d371520f773e2be9481142f79635e FV3/ccpp/framework (ccpp_transition_to_vlab_master_20190705-718-g741212e)
 1c2a898e97f069eeeb930cfb542db67ed349b72c FV3/ccpp/physics (EP4-744-g1c2a898e)
 74a0e098b2163425e4b5466c2dfcf8ae26d560a5 FV3/ccpp/physics/physics/Radiation/RRTMGP/rte-rrtmgp (v1.6)
 be0410ece28f2b5b9c089f8ca09ce0c80c79fe6c FV3/upp (upp_v10.2.0-191-gbe0410e)
-1ba8270870947b583cd51bc72ff8960f4c1fb36e FV3/upp/sorc/libIFI.fd
-7476b8f2790a47d788f79cebfdbb551567ae7cf8 FV3/upp/sorc/ncep_post.fd/post_gtg.fd
 041422934cae1570f2f0e67239d5d89f11c6e1b7 GOCART (sdr_v2.1.2.6-119-g0414229)
 35789c757766e07f688b4c0c7c5229816f224b09 HYCOM-interface/HYCOM (2.3.00-121-g35789c7)
 3bcfbbea8b865c3f88497049fde044c1145bd202 MOM6-interface/MOM6 (dev/master/repository_split_2014.10.10-10050-g3bcfbbea8)
 9423197f894112edfcb1502245f7d7b873d551f9 MOM6-interface/MOM6/pkg/CVMix-src (9423197)
 29e64d652786e1d076a05128c920f394202bfe10 MOM6-interface/MOM6/pkg/GSW-Fortran (29e64d6)
 1e259014c1eba9070cec7027d8b4b479ae54275a NOAHMP-interface/noahmp (v3.7.1-430-g1e25901)
 d9b3172f4197c65d471662c6952a668152d71230 WW3 (6.07.1-345-gd9b3172f)
 31e4e3e57bad8a7cdaa0ce2d3c4efbca63b5f74a stochastic_physics (ufs-v2.0.0-216-g31e4e3e)

NOTES:
[Times](Memory) are at the end of each compile/test in format [MM:SS](Size).
The first time is for the full script (prep+run+finalize).
The second time is specifically for the run phase.
Times/Memory will be empty for failed tests.

BASELINE DIRECTORY: /lfs/h2/emc/nems/noscrub/emc.nems/RT/NEMSfv3gfs/develop-20240503
COMPARISON DIRECTORY: /lfs/h2/emc/ptmp/brian.curtis/FV3_RT/rt_17831

RT.SH OPTIONS USED:
* (-a) - HPC PROJECT ACCOUNT: GFS-DEV
* (-n) - RUN SINGLE TEST: cpld_warmstart_c48
* (-e) - USE ECFLOW

PASS -- COMPILE 's2s_intel' [09:21, 09:05] ( 1 warnings 1 remarks )
PASS -- TEST 'cpld_warmstart_c48_intel' [54:48, 01:29](2702 MB)

SYNOPSIS:
Starting Date/Time: 20240515 17:55:41
Ending Date/Time: 20240515 18:10:27
Total Time: 00h:14m:53s
Compiles Completed: 1/1
Tests Completed: 1/1

NOTES:
A file 'test_changes.list' was generated but is empty.
If you are using this log as a pull request verification, please commit 'test_changes.list'.

Result: SUCCESS

====END OF ACORN REGRESSION TESTING LOG====

the E in PBS is not for "error" but "Exiting" and caused the ecflow job to terminate before finishing even when the model did finish

DusanJovic-NOAA commented 1 day ago
====START OF ACORN REGRESSION TESTING LOG====

UFSWM hash used in testing:
16f3271ba7e5266662904d1524d87a80f97bf9bf

Submodule hashes used in testing:
 37cbb7d6840ae7515a9a8f0dfd4d89461b3396d1 AQM (v0.2.0-37-g37cbb7d)
 be5d28fd1b60522e6fc98aefeead20e6aac3530b AQM/src/model/CMAQ (CMAQv5.2.1_07Feb2018-198-gbe5d28fd1)
 3d7067a523b8557058755289e4275f5f5c985daf CDEPS-interface/CDEPS (cdeps0.4.17-40-g3d7067a)
 9c0cd8d3ba5963d01711094a3c81b6358fcb8367 CICE-interface/CICE (CICE6.0.0-448-g9c0cd8d)
 f6ff8f7c4d4cb6feabe3651b13204cf43fc948e3 CICE-interface/CICE/icepack (Icepack1.1.0-182-gf6ff8f7)
 4e19850cb083bc474b7cde5dc2f8506ec74cc442 CMEPS-interface/CMEPS (cmeps_v0.4.1-2306-g4e19850)
 cabd7753ae17f7bfcc6dad56daf10868aa51c3f4 CMakeModules (v1.0.0-28-gcabd775)
 b329c2ccc4c5c6936cb1f548eef83d2a1ee36ae2 FV3 (remotes/origin/hydro)
 aabfdccae440da7389195762601db7fcc09f6f11 FV3/atmos_cubed_sphere (remotes/origin/hydro)
 741212e4962d371520f773e2be9481142f79635e FV3/ccpp/framework (ccpp_transition_to_vlab_master_20190705-718-g741212e)
 1c2a898e97f069eeeb930cfb542db67ed349b72c FV3/ccpp/physics (EP4-744-g1c2a898e)
 74a0e098b2163425e4b5466c2dfcf8ae26d560a5 FV3/ccpp/physics/physics/Radiation/RRTMGP/rte-rrtmgp (v1.6)
 be0410ece28f2b5b9c089f8ca09ce0c80c79fe6c FV3/upp (upp_v10.2.0-191-gbe0410e)
-1ba8270870947b583cd51bc72ff8960f4c1fb36e FV3/upp/sorc/libIFI.fd
-7476b8f2790a47d788f79cebfdbb551567ae7cf8 FV3/upp/sorc/ncep_post.fd/post_gtg.fd
 041422934cae1570f2f0e67239d5d89f11c6e1b7 GOCART (sdr_v2.1.2.6-119-g0414229)
 35789c757766e07f688b4c0c7c5229816f224b09 HYCOM-interface/HYCOM (2.3.00-121-g35789c7)
 3bcfbbea8b865c3f88497049fde044c1145bd202 MOM6-interface/MOM6 (dev/master/repository_split_2014.10.10-10050-g3bcfbbea8)
 9423197f894112edfcb1502245f7d7b873d551f9 MOM6-interface/MOM6/pkg/CVMix-src (9423197)
 29e64d652786e1d076a05128c920f394202bfe10 MOM6-interface/MOM6/pkg/GSW-Fortran (29e64d6)
 1e259014c1eba9070cec7027d8b4b479ae54275a NOAHMP-interface/noahmp (v3.7.1-430-g1e25901)
 d9b3172f4197c65d471662c6952a668152d71230 WW3 (6.07.1-345-gd9b3172f)
 31e4e3e57bad8a7cdaa0ce2d3c4efbca63b5f74a stochastic_physics (ufs-v2.0.0-216-g31e4e3e)

NOTES:
[Times](Memory) are at the end of each compile/test in format [MM:SS](Size).
The first time is for the full script (prep+run+finalize).
The second time is specifically for the run phase.
Times/Memory will be empty for failed tests.

BASELINE DIRECTORY: /lfs/h2/emc/nems/noscrub/emc.nems/RT/NEMSfv3gfs/develop-20240503
COMPARISON DIRECTORY: /lfs/h2/emc/ptmp/brian.curtis/FV3_RT/rt_17831

RT.SH OPTIONS USED:
* (-a) - HPC PROJECT ACCOUNT: GFS-DEV
* (-n) - RUN SINGLE TEST: cpld_warmstart_c48
* (-e) - USE ECFLOW

PASS -- COMPILE 's2s_intel' [09:21, 09:05] ( 1 warnings 1 remarks )
PASS -- TEST 'cpld_warmstart_c48_intel' [54:48, 01:29](2702 MB)

SYNOPSIS:
Starting Date/Time: 20240515 17:55:41
Ending Date/Time: 20240515 18:10:27
Total Time: 00h:14m:53s
Compiles Completed: 1/1
Tests Completed: 1/1

NOTES:
A file 'test_changes.list' was generated but is empty.
If you are using this log as a pull request verification, please commit 'test_changes.list'.

Result: SUCCESS

====END OF ACORN REGRESSION TESTING LOG====

the E in PBS is not for "error" but "Exiting" and caused the ecflow job to terminate before finishing even when the model did finish

Hmm. This was never an issue. Before the recent 'cleanup' code that handled the 'E' status looked like:

      elif [[ $status = 'E' ]] || [[ $status = 'C' ]] || [[ $status = '-' ]];  then
        status_label='finished'
        test_status='DONE'
        exit_status=$( qstat ${jobid} -x -f | grep Exit_status | awk '{print $3}')
        if [[ $exit_status != 0 ]]; then
          test_status='FAIL'
        fi
      else
        status_label='finished'
      fi

That part of code has been completely rewritten. I wonder how many more subtle error like this we are going to find in the future.

For example the code that checks the 'Exit_status' of the finished job:

     exit_status=$( qstat ${jobid} -x -f | grep Exit_status | awk '{print $3}')
        if [[ $exit_status != 0 ]]; then
          test_status='FAIL'
        fi

is completely removed. Why?

DusanJovic-NOAA commented 1 day ago

Also, this logic has been removed:

      else
        echo "Slurm unknown status ${status}. Check sacct ..."
        sacct -n -j ${slurm_id} --format=JobID,state%20,Jobname%20
        status_label=$( sacct -n -j ${slurm_id} --format=JobID,state%20,Jobname%20 | grep "^${slurm_id}" | grep ${JBNME} | awk '{print $2}' )
        if [[ $status_label = 'FAILED' ]] || [[ $status_label = 'TIMEOUT' ]] || [[ $status_label = 'CANCELLED' ]] ; then
            test_status='FAIL'
        fi
      fi
BrianCurtis-NOAA commented 1 day ago

@DusanJovic-NOAA I don't think any of this pertains to this PR, we can move forward with this PR at this point.

Please feel free to start an e-mail discussion on recent rt.sh and rt_utils.sh changes.

zach1221 commented 1 day ago

@XiaqiongZhou-NOAA Fv3atm merged. Please update submodule hash and revert change to .gitmodule url. Hash: https://github.com/NOAA-EMC/fv3atm/commit/9534e14d3c740c8aa9c915ad03f36dc7b75d8ab8

XiaqiongZhou-NOAA commented 1 day ago

done.

On Wed, May 15, 2024 at 4:38 PM zach1221 @.***> wrote:

@XiaqiongZhou-NOAA https://github.com/XiaqiongZhou-NOAA Fv3atm merged. Please update submodule hash and revert change to .gitmodule url. Hash: @.*** https://github.com/NOAA-EMC/fv3atm/commit/9534e14d3c740c8aa9c915ad03f36dc7b75d8ab8

— Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-weather-model/pull/2255#issuecomment-2113416581, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALQE7UX2AAEHTIACLDPFGVTZCPBULAVCNFSM6AAAAABG22ZRWGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJTGQYTMNJYGE . You are receiving this because you were mentioned.Message ID: @.***>

zach1221 commented 1 day ago

@DeniseWorthen @NickSzapiro-NOAA can you please review your unresolved conversation?

NickSzapiro-NOAA commented 1 day ago

@DeniseWorthen @NickSzapiro-NOAA can you please review your unresolved conversation?

@zach1221 I marked my comment as resolved. I don't see any other options... This PR still needs commit messages, fwiw

zach1221 commented 1 day ago

@XiaqiongZhou-NOAA I think Fernando mentioned this previously, but can you please fill out the commit message section of the PR template, which summarizes briefly your code changes included in this PR and sub-component PRs?

XiaqiongZhou-NOAA commented 1 day ago

@XiaqiongZhou-NOAA I think Fernando mentioned this previously, but can you please fill out the commit message section of the PR template, which summarizes briefly your code changes included in this PR and sub-component PRs?

Sorry, I did not notice. Commit message is added.