weecology / bbs-forecasting

Research on forecasting using Breeding Bird Survey data
MIT License
7 stars 9 forks source link

Cite Ward et al. #190

Closed ethanwhite closed 7 years ago

ethanwhite commented 7 years ago

For both time-series for forecasting in ecology and probably noting this, which it looked like we had except for the observer model: "the most accurate model was one that simply treated the most recent observation as the forecast"

ethanwhite commented 7 years ago

Also possibly as a "but see" based on "and that estimation of the observation error variance also comes at a high cost, an issue also discussed by Holmes et al. (2007)."

ethanwhite commented 7 years ago

Given the take message of this paper we should probably directly discuss and cite it in relation to our baseline results. Very similar result. Last paragraph:

The baseline model used in our analysis was a simple random walk without drift. For this model, the t-step ahead forecast is simply the last observed value. No additional model parameters are estimated for the actual forecast, though the calculation of the ASE (the prediction error) uses an estimate of the total variance (as do all models). The failure of the more complicated time-series models to provide short-term predictions with lower error than the random walk without drift emphasizes 1) the cost of estimating parameters in the face of noise and 2) the cost of basing short-term predictions on parameters, like the trend over the whole time series, which may be more associated with long-term dynamics rather than short-term behavior. For short population time series, we can recommend the use of more complex forecasting models only when time series have strong internal structure (e.g. the cyclic dynamics in salmon) or have lower variability and higher temporal autocorrelation (larger species with higher maximum ages or higher trophic level). In summary, fitting models with many parameters and the flexibility to model complex structure may be tempting, but this involves estimating structure from few data points. We found that estimation of even one or two parameters imposes a high cost with little benefit for short-term forecasts of population abundance for species without obvious cyclic population dynamics.

sdtaylor commented 7 years ago

So something like this, probably in paragraph 5 of the discussion?

Ward et al. (2014) found similar patterns in time series of fisheries stocks, where relatively stable time series were best predicted by simple models and more complex models were only beneficial with dynamic time series.