epag
commented
3 weeks ago Original Redmine Comment Author Name: Xia (Xia) Original Date: 2018-03-06T20:00:33Z
James:
Thanks for putting out a nice description of peak analysis.
Regarding to your recommendations in the numbered Bulletins:
- I concur with you on this suggestion, that is to use all data to identify the peak and focus on the paired flow time series that cover the peak range.
- Do you have a specific low-pass filter in mind for detecting non-local peak? Or is it available in the WRES?
- A review of peak detection method would be very helpful. Have you searched the relevant literature, and if so, could you please share them with us? I also talked with Alex about the peak derivation method used by the MBRFC folks shown in the attached Excel file. The general concept is similar to what you mentioned: subset paired time-series and obtain threshold-based peaks if flow exceeds a threshold stage, such as forecast issuance, minor-, moderate-, major-flood and record stage. For each threshold level, the time-to-peak of forecasted AHPS locations within select watersheds stratified in terms of fast, moderate and slow response time are averaged to obtain one value representative of three categories of basins with different response times.
Best regards,
Xia
Author Name: James (James) Original Redmine Issue: 46360, https://vlab.noaa.gov/redmine/issues/46360 Original Date: 2018-02-08
Given a set of paired time-series (of predictions and verifying observations), timing error metrics may be applied to the unconditional set of time-series or to a subset of time-series that meet specific conditions. A subset may be defined in a variety of ways, such as:
This feature request is concerned with identifying paired time-series that contain a "peak" or a "peak above a threshold".
An example use case involves the calculation of time-to-peak errors, in order to evaluate the quality of a hydrologic forecast model at predicting the timing of flood peaks. Currently, the time-to-peak errors are computed for the unconditional set of paired time-series (i.e. all data). While this may provide some indication of timing errors in general, it will not specifically focus on time-series that contain peaks above a flood threshold. Thus, for this use case, a better indication of timing errors is likely to be obtained from the subset of paired time-series that contain a peak above the flood threshold.
Identifying a "local peak" or "high frequency peak" is an objective activity, since a local peak represents a stationary point or maximal turning point in some time-varying variable, such as streamflow. In simple terms, a local peak occurs where the value either side of the peak is lower than the value at the peak.
An example of local peak detection is provided by the @findpeaks@ method in Matlab.
https://uk.mathworks.com/help/signal/ref/findpeaks.html
However, local peak detection is likely to have only limited value in this context, since environmental variables, such as streamflow, are noisy variables. In other words, they typically contain a very large number of local peaks, depending on the scale of the data. For example, instantaneous data is inherently more noisy than daily aggregated data and will contain a large number of local peaks. For hydrologists, low-frequency peaks are typically more interesting, because they correspond to hydrologically relevant behavior, such as a flood wave.
"Non-local" peak detection is fundamentally more challenging than local peak detection for several reasons. In particular, when a time-series is truncated (e.g. by a forecast horizon), it will often be unclear whether that time-series contains a non-local peak. In the following example, @x@ might reasonably be identified as a non-local peak in scenario @(c)@, but not in scenarios @(a)@ and @(b)@.
!Peak.PNG!
Unless the time-horizon is augmented by additional data (e.g. by looking at observations or simulations across a larger time horizon), a non-local peak can only be defined in the context of the time horizon available.
The first question that arises, therefore, is whether non-local peaks should be identified from the paired data or from some other data, such as observations or hydrologic simulations that cover a longer period (including the period of the paired data).
A second question that arises concerns the definition of "non-local". Non-local could mean "global", for example, or "regional" or something else. A global peak is not simply the local peak with the maximum value of all local peaks. Rather, a "global peak" is concerned with "looking through" local peaks in the data to find a low-frequency pattern that can be identified as something more important (e.g. like detecting a "flood peak" from a noisy time-series of instantaneous streamflow).
Central to the definition of "non local" is signal processing theory or signal detection theory, for which an enormous body of literature exists.
One possibility for identifying "non local" peaks would be to apply a low-pass filter across the original data or to transform the signal into the frequency domain (Fast Fourier Transform) and remove the high-frequency components before recomposing the signal. Once filtered, a local peak detection algorithm could be applied. The degree and type of filtering would need to be determined and would be necessarily subjective, at least to some degree.
To make progress on this, we will probably need to identify a simple approach that can be augmented with better approaches in future. My preliminary recommendations are that we:
This is a high priority task, that will need an initial implementation for SR1.
I am adding Xia and Yuqiong as watchers, as they are likely to have valuable input on a possible first approach for selecting time-series that contain relevant "peaks" for verification. We may want to add other watchers in due course (e.g. IOEP folks).
Redmine related issue(s): 44781, 44801, 70527