Combining forecasts from auto.arima, ets, and tbats

[dashaub]

This post and a project I'm working on for generating many forecasts in an automated manner has me thinking about selecting between ets(), auto.arima(), and tbats() models in an automated manner and possibly combining the forecasts into an ensemble for a better forecast. http://ellisp.github.io/blog/2016/01/30/hybrid-forecasts/

tl;dr summary Point forecasts obtained by weighting ets() and auto.arima() point forecasts with 1/2 weights are more accurate than point forecasts from either individual method. Prediction intervals tend to overstate the confidence of future values, and using the max() from the upper confidence bound and min() from the lower confidence bound from the individual arima and ets forecasts produces confidence intervals that come closer to matching their purported inclusion certainty for future values.

• Would you consider including a function similar to hybridf() from the blog? I envision it with the additional ability to include forecasts from specified combinations of auto.arima(), ets(), and tbats() forecasts.
• The downside of this that I see is while this approach seems to work well empirically on these datasets, the ensemble model is not well grounded in a statistical specification, unlike ets/arima/tbats. Furthermore, the equal weights from each model for the point forecast and min/max construction of the confidence bound are almost certainly not optimal.
• An approach I see to rectify this would be to run crossvalidation on the series for each model to obtain out-of-sample error measures. These error measures would be used for weighting the models and the confidence limits in the ensemble. This approach would of course be more computationally costly. Perhaps there would be an argument in the function hybridf() to enable this or alternatively use the quicker method above?

In general I'd be eager to read any literature recommendations you might have about optimal combination of forecasts generated from various methods.

[ellisp]

hybridf() was coded by me in a fork of {forecast} and I'd be very happy to submit a PR after any feedback / suggestions, if @robjhyndman is interested.

[robjhyndman]

I'd rather wait until the function had some academic support in the form of a published paper backing the particular combination and construction. Alternatively, it could be part of a new package with forecast as a dependency.

There is quite a literature on forecast combination, including the frequent observation that simple averaging is usually better than weighted averaging (known as the "forecast combination puzzle"). See http://www.sciencedirect.com/science/article/pii/S016920701200088X for a recent discussion.