dtcenter / METcalcpy

https://metcalcpy.readthedocs.io/en/latest/index.html
Apache License 2.0
16 stars 2 forks source link

Add support for the Forecast Convergence Score to MET. #49

Open JohnHalleyGotway opened 4 years ago

JohnHalleyGotway commented 4 years ago

This is a request from NOAA/MDL: https://sats.nws.noaa.gov/~verification/mos/help.html

Currently applied to point locations, not gridded analyses. But it could be applied to gridded analyses.

This is related to forecast consistency and should be included as one of the statistics.

Forecast Convergence Score (FCS)

The Ruth-Glahn Forecast Convergence Score (FCS) is an index that measures the number of large swings in forecasts made over a series of forecast cycles for forecasts valid at the same time. Assume there are N forecasts made over a number of days. With each subsequent forecast cycle, the forecast projection decreases until the valid time of the forecast is reached. The FCS is defined as follows:

(T1 + T2) FCS = ----------------- (T3 + T4)

where T1 = the number of forecasts that changed insignificantly (less than the threshold) from the previous forecast Fi-1 OR moved closer to the next forecast Fi+1, where i varies from 2 to N. When i=N, the observation is used as the next forecast Fi+1.

|FN - F1| T2 = ------------------- threshold

T3 = N-1

|Fi - Fi-1| (from i = 2 to N) T4 = ---------------------- threshold

The T1 and T3 terms account for the actual and possible number of swings, respectively.

The T2 and T4 terms account for the magnitude of the swings.

A significance threshold specifies the minimum change necessary to count as a swing.

The FCS ranges from 0.0 (many large swings with no convergence toward the observation) to 1.0 (no large swings with the forecasts converging toward the observation.)

A "case" is an unbroken series of forecasts for the entire 7 days, all valid at the same time, ending with the observation at the valid time. Thus, the number of cases is generally smaller than the number of cases for other statistics. Any gap in the forecasts or observations causes the case to be eliminated from the sample.

JohnHalleyGotway commented 4 years ago

Need to define where this functionality should be implemented: MET, METviewer, METcalcpy/METplotpy

JohnHalleyGotway commented 3 years ago

This was originally defined in MET but belongs in METcalcpy instead. This is related to forecast consistency.