Closed ethanwhite closed 7 years ago
Also possibly as a "but see" based on "and that estimation of the observation error variance also comes at a high cost, an issue also discussed by Holmes et al. (2007)."
Given the take message of this paper we should probably directly discuss and cite it in relation to our baseline results. Very similar result. Last paragraph:
The baseline model used in our analysis was a simple random walk without drift. For this model, the t-step ahead forecast is simply the last observed value. No additional model parameters are estimated for the actual forecast, though the calculation of the ASE (the prediction error) uses an estimate of the total variance (as do all models). The failure of the more complicated time-series models to provide short-term predictions with lower error than the random walk without drift emphasizes 1) the cost of estimating parameters in the face of noise and 2) the cost of basing short-term predictions on parameters, like the trend over the whole time series, which may be more associated with long-term dynamics rather than short-term behavior. For short population time series, we can recommend the use of more complex forecasting models only when time series have strong internal structure (e.g. the cyclic dynamics in salmon) or have lower variability and higher temporal autocorrelation (larger species with higher maximum ages or higher trophic level). In summary, fitting models with many parameters and the flexibility to model complex structure may be tempting, but this involves estimating structure from few data points. We found that estimation of even one or two parameters imposes a high cost with little benefit for short-term forecasts of population abundance for species without obvious cyclic population dynamics.
So something like this, probably in paragraph 5 of the discussion?
Ward et al. (2014) found similar patterns in time series of fisheries stocks, where relatively stable time series were best predicted by simple models and more complex models were only beneficial with dynamic time series.
For both time-series for forecasting in ecology and probably noting this, which it looked like we had except for the observer model: "the most accurate model was one that simply treated the most recent observation as the forecast"