Feature/mesoscale beta5

PerryShafran-NOAA commented 1 month ago

Note to developers: You must use this PR template!

Description of Changes

Please include a summary of the changes and the related GitHub issue(s). Please also include relevant motivation and context.

This PR updates the following:

1) Uses the beta5 versions of MET and METplus to run the stats and plots for the mesoscale NAM and RAP grid2obs runs.
2) Updates the plotting code to ensure that the last date on the plot matches with the VDATE of the plot tarball. 3) Updates the CAPE code to use the INTERSECTION rather than the UNION, as in PR #567 for the cam updates.

Developer Questions and Checklist

Is this a high priorty PR? If so, why and is there a date it needs to be merged by?

Not a high priority PR, but important so we can move the parallels to beta5.

Do you have any planned upcoming annual leave/PTO?

Will be off for Rosh Hashanah on Thursday/Friday 10/3 to 10/4.

Are there any changes needed for when the jobs are supposed to run?

No.

[x ] The code changes follow NCO's EE2 Standards.
[x ] Developer's name is removed throughout the code and have used ${USER} where necessary throughout the code.
[x ] References the feature branch for HOMEevs are removed from the code.
[ x] J-Job environment variables, COMIN and COMOUT directories, and output follow what has been defined for EVS.
[ ] Jobs over 15 minutes in runtime have restart capability. - NO, next mesoscale PR.
[x ] If applicable, changes in the dev/drivers/scripts or dev/modulefiles have been made in the corresponding ecf/scripts and ecf/defs/evs-nco.def?
[ x] Jobs contain the approriate file checking and don't run METplus for any missing data.
[x] Code is using METplus wrappers structure and not calling MET executables directly.
[x ] Log is free of any ERRORs or WARNINGs.

Testing Instructions

Please include testing instructions for the PR assignee. Include all relevant input datasets needed to run the tests.

For mesoscale stats:

1) Clone my fork. 2) checkout feature/mesoscale_beta5 3) Go to the directory dev/drivers/scripts/stats/mesoscale 4) Edit files jevs_mesoscale_nam_grid2obs_stats.sh and jevs_mesoscale_rap_grid2obs_stats.sh 5) Set HOMEevs to the working directory. 6) Set export EVSINspcotlk=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/evs_beta5/v2.0/prep/cam 7) Submit the following files using this nomenclature.

qsub -v vhr=07 jevs_mesoscale_nam_grid2obs_stats.sh qsub -v vhr=07 jevs_mesoscale_rap_grid2obs_stats.sh

For mesoscale plots:

1) Go to the directory dev/drivers/scripts/plots/mesoscale. 2) Go to the file jevs_mesoscale_grid2obs_plots.sh 3) Set HOMEevs to the working directory. 4) Set export COMIN=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/evs_beta5/$evs_ver_2d 5) Submit by doing the following:

qsub -v vhr=01 jevs_mesoscale_grid2obs_plots.sh

malloryprow commented 1 month ago

Ah! Sorry for the delay @PerryShafran-NOAA. I hadn't realized this one was moved from draft to ready. I'll review the 3 changed files and then start setting up testing.

malloryprow commented 1 month ago

Is there a reason that mesoscale needs export EVSINspcotlk=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/evs_beta5/v2.0/prep/cam set?

PerryShafran-NOAA commented 1 month ago

@malloryprow I thought that without it set that it was pointing to a directory in perry.shafran that didn't exist, but maybe I didn't have COMIN correctly set at the time. Let me see what happens without that set.

PerryShafran-NOAA commented 1 month ago

@malloryprow I did not set NET to evs_beta5 nor did I set COMIN to emc.vpppg because mesoscale has no prep. I suppose you could just simply set COMIN to point to emc.vpppg/evs_beta5, and that would be all you'd need to do rather than set EVSINprep. I tried a number of combinations, and this is what I found had worked.

malloryprow commented 1 month ago

Sorry wasn't concerned about the location but more so what/why mesoscale stats needs something from cam prep?

AliciaBentley-NOAA commented 1 month ago

Looking at the mesoscale webpage, I do not see any verification being done with SPC outlooks or SPC storm reports. Is that correct, Perry/Andrew?

PerryShafran-NOAA commented 1 month ago

Oh! That's where the generated spcoutlook files are.

You'll see that is how it is set in the j-job:

export EVSINspcotlk=${EVSINspcotlk:-$COMIN/prep/cam}

malloryprow commented 1 month ago

Ah, I hadn't realized that mesoscale was verifying the SPC Outlook regions as well.

PerryShafran-NOAA commented 1 month ago

@AliciaBentley-NOAA I definitely see spc_otlk output files, primarily to verify SBCAPE and MLCAPE. I think there are CAPE plots, but maybe these stat files aren't plotted? Let me check.

PerryShafran-NOAA commented 1 month ago

@AliciaBentley-NOAA @AndrewBenjamin-NOAA @malloryprow Yes, spc outlook regions are used. There are files called, for example, point_stat_nam_spc_otlk_SBCAPE_OBS_660000L_20240930_120000V.stat, in the small stat archive. These are CAPE obs validated over the SPC Outlook Regions.

I guess they never made their way to the plot code?

malloryprow commented 1 month ago

Running the stats with vhr=07. COMOUT for both jobs is /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr571/evs/v2.0/stats/mesoscale.

jevs_mesoscale_nam_grid2obs_stats.sh

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr571/EVS/dev/drivers/scripts/stats/mesoscale/jevs_mesoscale_nam_grid2obs_stats_00.o156956962 DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_mesoscale_nam_grid2obs_stats_00.156956962.cbqs01

jevs_mesoscale_rap_grid2obs_stats.sh

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr571/EVS/dev/drivers/scripts/stats/mesoscale/jevs_mesoscale_rap_grid2obs_stats_00.o156956987 DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_mesoscale_rap_grid2obs_stats_00.156956987.cbqs01

AndrewBenjamin-NOAA commented 1 month ago

@AliciaBentley-NOAA @AndrewBenjamin-NOAA @malloryprow Yes, spc outlook regions are used. There are files called, for example, point_stat_nam_spc_otlk_SBCAPE_OBS_660000L_20240930_120000V.stat, in the small stat archive. These are CAPE obs validated over the SPC Outlook Regions.

I guess they never made their way to the plot code?

Per our offline discussion, I think we save this for a seperate PR and proceed with the beta5 testing.

PerryShafran-NOAA commented 1 month ago

@malloryprow All look good. There are no ERRORs. There is a WARNING about a deprecated METplus variable, but I am fixing that in the restart PR that I've already worked on, but that's not included here. There are no ERRORs. Difference in stat files in the beta5 emc.vpppg stat files are all CAPE related, which is expected due to the CAPE fix from Marcel's cam PR.

I would think that we could approve this PR.

AliciaBentley-NOAA commented 1 month ago

Cool! Can we potentially look at some plots from the beta5 tests (perhaps of CAPE) before merging this PR?

@AndrewBenjamin-NOAA @PerryShafran-NOAA @malloryprow

PerryShafran-NOAA commented 1 month ago

Oops!!! What I meant was, let's move on to the plots. Nearly forgot that plotting is part of this PR. I got ahead of myself. Sorry about that!!

malloryprow commented 1 month ago

Running plots!

jevs_mesoscale_grid2obs_plots.sh

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr571/EVS/dev/drivers/scripts/plots/mesoscale/jevs_mesoscale_grid2obs_plots.o156964238 COMOUT: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr571/evs/v2.0/plots/mesoscale DATA: =/lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_mesoscale_grid2obs_plots.156964238.cbqs01

PerryShafran-NOAA commented 1 month ago

@AliciaBentley-NOAA @AndrewBenjamin-NOAA @malloryprow I am posting these plots online here: https://www.emc.ncep.noaa.gov/mmb/pshaf/evs_plots_mesoscale/files.php

The plots are still transferring so give it a few minutes before all the png files transfer to the emcrzdm.

PerryShafran-NOAA commented 1 month ago

@AliciaBentley-NOAA @AndrewBenjamin-NOAA @malloryprow I now have a test mesoscale web page, now located here: https://emc.ncep.noaa.gov/users/verification/regional/mesoscale/test/

You can see all plots here; it's much faster than what I was trying to do above.

PerryShafran-NOAA commented 1 month ago

@malloryprow I am seeing the vector wind plot here. It looks good:

https://emc.ncep.noaa.gov/users/verification/regional/mesoscale/test/grid2obs/ugrd_vgrd10m/

malloryprow commented 1 month ago

I think it looks good too. Since beta5 parallel doesn't have a full 31 days of stats yet, the numbers aren't identical but values are similar.

AliciaBentley-NOAA commented 1 month ago

The CAPE observation fix included in this PR looks good, too! CC @MarcelCaron-NOAA to confirm. Not 100% sure why the GFS line is missing at the latest time, but at least the gray obs line is there!

evs mesoscale fbar_obar cape_l0 last31days timeseries_init00z_f012 buk_conus

malloryprow commented 1 month ago

The CAPE observation fix included in this PR looks good, too! CC @MarcelCaron-NOAA to confirm. Not 100% sure why the GFS line is missing at the latest time, but at least the gray obs line is there!

GFS stats for VDATE=20241001 won't start running until 1845Z!

AliciaBentley-NOAA commented 1 month ago

Makes sense! All looks good to me!

malloryprow commented 1 month ago

I think we are good to merge this pending your review @AndrewBenjamin-NOAA!

malloryprow commented 1 month ago

I concur, thanks @PerryShafran-NOAA!

malloryprow commented 1 month ago

Also, I think some items addressed here may be on the Fixes and Additions document so don't forget to ✔️ those off.

NOAA-EMC / EVS