Add support to Gen-Ens-Prod for computing probabilities using percentile thresholds.

[JohnHalleyGotway]

Describe the Enhancement

This is a request for an enhancement made via met-help on 12/21/2020: https://rt.rap.ucar.edu/rt/Ticket/ModifyAll.html?id=97897

This functionality was also requested by David Ruth from NOAA/MDL.

Brian Matilla from NOAA/NSSL requested that the Ensemble-Stat tool be enhanced to support the computation of probabilities using percentile thresholds. As of met-9.1, percentile thresholds are supported in the context of the obs_thresh setting. However, they are not supported for cat_thresh in the "ens" dictionary. This task is to add that support.

Note, that this it would only make sense to support forecast sample percentile (SFP) threshold types and user-specified (USP) thresholds, but not SOP, SCP, or FBIAS1 thresholds since we only have forecast data available in the "ens" dictionary. Also note that these percentiles should be computed using all ensemble members... not separately for each. And they should probably be computed over the entire model domain... and not separately for each masking region because no masking regions apply for to the "ens" dictionary.

Time Estimate

2 days?

Sub-Issues

Consider breaking the enhancement down into sub-issues. No sub-issues

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• [x] Select engineer(s) or no engineer required: John Halley Gotway
• [ ] Select scientist(s) or no scientist required

Labels

• [x] Select component(s)
• [x] Select priority
• [x] Select requestor(s)

Projects and Milestone

• [x] Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
• [x] Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

• [x] METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy No impacts.

Enhancement Checklist

See the METplus Workflow for details.

• [ ] Complete the issue definition above, including the Time Estimate and Funding Source.
• [ ] Fork this repository or create a branch of develop. Branch name: feature_<Issue Number>_<Description>
• [ ] Complete the development and test your changes.
• [ ] Add/update log messages for easier debugging.
• [ ] Add/update unit tests.
• [ ] Add/update documentation.
• [ ] Push local changes to GitHub.
• [ ] Submit a pull request to merge into develop. Pull request: feature <Issue Number> <Description>
• [ ] Define the pull request metadata, as permissions allow. Select: Reviewer(s), Project(s), Milestone, and Linked issues
• [ ] Iterate until the reviewer(s) accept and merge your changes.
• [ ] Delete your fork or branch.
• [ ] Close this issue.

[JohnHalleyGotway]

Met with NOAA/GSL on 9/30/21 to review ensemble verification plans, and @JeffHamiltonNOAA emphasized the need for more broad support of percentile thresholds in ensemble verification, especially those computed relative to the input ensemble member values.

[hertneky]

@JohnHalleyGotway Percentile thresholding within gen_ens_prod would be a useful tool for evaluating ensemble wind forecasts for a project with the Idaho Power Company. While the current project funding is over at the end of December, it may go another year if the proposal is funded. In that case, this functionality would be desired.