NOAA-EMC / UPP

Other
36 stars 98 forks source link

RRFSv1.0 Product cleanup: removal of retired products from fv3lam_rrfs.xml #909

Closed AndrewBenjamin-NOAA closed 6 months ago

AndrewBenjamin-NOAA commented 6 months ago

The following products were removed from fv3lam_rrfs.xml. They are no longer needed/will be retired in RRFSv1.0.

REFZR: Equivalent radar reflectivity factor from rain REFZI: Equivalent radar reflectivity factor for snow PBLREG Planetary Boundary Layer Regime NCPCP Large-Scale Precipitation (non-convective) LSPA Land Surface Precipitation Accumulation WATR Water Runoff GWLOWS Group Water Lower Storage CDLYR Non-Convective Cloud Cover BMIXL Blackadars Mixing Length Scale POT Potential Temperature: 10 m above ground SPFH Specific Humidity: 10m above ground SWHR Solar Radiative Heating Rate UFLX Momentum Flux, U-Component VFLX Momentum Flux, V-Component LWHR Long-Wave Radiative Heating Rate BGRUN Baseflow-Groundwater Runoff EVP SNFALB TMP:sigma levels PRES:heights above ground level of 10 m and above SPFH:heights above ground level of 10 m and above SNOWC: Remove from native level file (all zero values) TCDC: Remove sigma levels on Native file HGT: Convective Cloud Top ACPCP VVEL: All pressure levels on prslev file: 250, 500, 700, 800mb on native level RCS RCSOL RCQ RCT

WCOSS2 tests show that products were removed from PRSLEV and NATLEV files. The test can be found here: /lfs/h2/emc/ptmp/andrew.benjamin/post_rrfs_2023062800.54503

WenMeng-NOAA commented 6 months ago

@AndrewBenjamin-NOAA Please open an UPP issue linking to this PR.

WenMeng-NOAA commented 6 months ago

@AndrewBenjamin-NOAA Please open an UPP issue linking to this PR.

@AndrewBenjamin-NOAA Never mind. I see your issue.

WenMeng-NOAA commented 6 months ago

@AndrewBenjamin-NOAA Please sync your branch with the latest UPP branch.

AndrewBenjamin-NOAA commented 6 months ago

@WenMeng-NOAA my branch has synched with the latest UPP develop branch

WenMeng-NOAA commented 6 months ago

@AndrewBenjamin-NOAA, @HuiyaChuang-NOAA found that the SSRUN entry in PRSLEV dataset and NALET dataset have inconsistent precision settings. Could you combine a fix in RRFS control file fv3lam_rrfs.xml as:

Change the entry stating line 2309
  <param>
       <shortname>ACM_SNOM_ON_SURFACE</shortname>
       <pname>SNOM</pname>
       <scale>4.0</scale>
       </param>
into
<param>
       <shortname>ACM_SNOM_ON_SURFACE</shortname>
       <pname>SNOM</pname>
       <scale>6.0</scale>
       </param>
AndrewBenjamin-NOAA commented 6 months ago

@WenMeng-NOAA done.

WenMeng-NOAA commented 6 months ago

@AndrewBenjamin-NOAA Can you also regenerate postxconfig-NT-fv3lam_rrfs.txt?

AndrewBenjamin-NOAA commented 6 months ago

@WenMeng-NOAA sorry about that,, I generated and committed the flat file.

WenMeng-NOAA commented 6 months ago

@AndrewBenjamin-NOAA Please verify my test /u/wen.meng/ptmp/fv3r_2023062800 on Dogwwod to ensure number of record changes in datasets NATLEV and PRSLVE for RRFS are expected.

MatthewPyle-NOAA commented 6 months ago

@AndrewBenjamin-NOAA I just opened an issue about LTNG data to see if it could be added for the PRSLEV files. Not sure if it would be easy to fold in this request in this PR, but throwing the idea out there.

WenMeng-NOAA commented 6 months ago

@FernandoAndrade-NOAA Please start the UPP RTs on Hera and Orion. There will be baseline updates for fv3r and 3drtma with this PR.

WenMeng-NOAA commented 6 months ago

The UPP RTs were completed on WCOSS2 and Hera. @FernandoAndrade-NOAA Please refer to my testing on Hera at /home/Wen.Meng/stmp2: Wen.Meng@hfe11 stmp2$ ls -ltr /diff -rw-r--r-- 1 Wen.Meng stmp 75 Apr 4 13:52 rap_2020072316_pe_test/WRFPRS.GrbF16.diff -rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:53 fv3r_2023062800/PRSLEV10.tm00.diff -rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:54 fv3r_2023062800_pe_test/PRSLEV10.tm00.diff -rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400_pe_test/NATLEV00.tm00.diff -rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400/NATLEV00.tm00.diff -rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800/NATLEV10.tm00.diff -rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400_pe_test/PRSLEV00.tm00.diff -rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800_pe_test/NATLEV10.tm00.diff -rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400/PRSLEV00.tm00.diff

AndrewBenjamin-NOAA commented 6 months ago

@WenMeng-NOAA I apologize for the delay. I am still seeing LSPA and SNOWC on the NATLEV file. Those entries have been removed from fv3lam_rrfs.xml and the postxconfig-NT-fv3lam_rrfs.txt files in my commit, so I am unsure why they are still there.

Also, @MatthewPyle-NOAA asked that I add LTNG to the PRSLEV file, as per issue #916. May I add that product to this PR?

WenMeng-NOAA commented 6 months ago

@WenMeng-NOAA I apologize for the delay. I am still seeing LSPA and SNOWC on the NATLEV file. Those entries have been removed from fv3lam_rrfs.xml and the postxconfig-NT-fv3lam_rrfs.txt files in my commit, so I am unsure why they are still there.

Also, @MatthewPyle-NOAA asked that I add LTNG to the PRSLEV file, as per issue #916. May I add that product to this PR?

@AndrewBenjamin-NOAA Thanks for the update. You might be focused on verifying the test. I will submit another PR to address issue #916.

WenMeng-NOAA commented 6 months ago

@AndrewBenjamin-NOAA In your fv3lam_rrfs.xml, the LSPA entry starts at line 2763 and the SNOWC entry starts at line 3519.

AndrewBenjamin-NOAA commented 6 months ago

@WenMeng-NOAA I just submitted a commit to address this. I must have missed it when I was addressing conflicts. This has fixed the issue, you may proceed.

WenMeng-NOAA commented 6 months ago

@WenMeng-NOAA I just submitted a commit to address this. I must have missed it when I was addressing conflicts. This has fixed the issue, you may proceed.

@AndrewBenjamin-NOAA We will wrap your PR for final testing.

WenMeng-NOAA commented 6 months ago

@AndrewBenjamin-NOAA I assume my testing on 04/02 was fine, except SNOWC and LSPA, correct?

AndrewBenjamin-NOAA commented 6 months ago

@WenMeng-NOAA yes that is correct

FernandoAndrade-NOAA commented 6 months ago

The UPP RTs were completed on WCOSS2 and Hera. @FernandoAndrade-NOAA Please refer to my testing on Hera at /home/Wen.Meng/stmp2:

Wen.Meng@hfe11 stmp2$ ls -ltr */*diff
-rw-r--r-- 1Wen.Meng stmp 75 Apr 4 13:52 rap_2020072316_pe_test/WRFPRS.GrbF16.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:53 fv3r_2023062800/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:54 fv3r_2023062800_pe_test/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400_pe_test/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400_pe_test/PRSLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800_pe_test/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400/PRSLEV00.tm00.diff

Hera and Orion RTs match your results, Hercules had different changes in fv3r/PRSLEV, rtma/NATLEV, and rtma/PRSLEV if you could review at /work2/noaa/epic/nandoam/regression-testing/upp/hercules/909/UPP/ci/rundir/upp-HERCULES

I did not see any issues in the run itself on Hercules.

WenMeng-NOAA commented 6 months ago

The UPP RTs were completed on WCOSS2 and Hera. @FernandoAndrade-NOAA Please refer to my testing on Hera at /home/Wen.Meng/stmp2:

Wen.Meng@hfe11 stmp2$ ls -ltr */*diff
-rw-r--r-- 1Wen.Meng stmp 75 Apr 4 13:52 rap_2020072316_pe_test/WRFPRS.GrbF16.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:53 fv3r_2023062800/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:54 fv3r_2023062800_pe_test/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400_pe_test/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400_pe_test/PRSLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800_pe_test/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400/PRSLEV00.tm00.diff

Hera and Orion RTs match your results, Hercules had different changes in fv3r/PRSLEV, rtma/NATLEV, and rtma/PRSLEV if you could review at /work2/noaa/epic/nandoam/regression-testing/upp/hercules/909/UPP/ci/rundir/upp-HERCULES

I did not see any issues in the run itself on Hercules.

The tests on Hercules look good to me.

WenMeng-NOAA commented 6 months ago

This PR is ready for merging.