epiforecasts / EpiSoon

Forecasting the effective reproduction number over short timescales
https://epiforecasts.io/EpiSoon/
Other
7 stars 3 forks source link

Improve the handling of uncertainty when using `forecastHybrid` #64

Open seabbs opened 4 years ago

seabbs commented 4 years ago

The forecastHybrid package can produce some robust cross-validation weighted ensembles. Unfortunately, it handles confidence intervals by taking an unweighted mean of all candidate models or using the max/min for the candidate models.

As we only use samples from models (except when samples = 1 and a point forecast is used) this behaviour is a problem. In the current wrapper forecastHybrid_model we take samples by assuming the forecast is normally distributed and reconstructing a mean and standard deviation from the upper and lower bounds. This means that we see no benefit of any kind from a weighted ensemble. In an ideal scenario, the underlying package would produce better samples for its forecasts but failing that a more elegant workaround would be ideal.