Bootstrapped prediction intervals

[nredell]

This is long overdue. There are residuals waiting around to be boostrapped and turned into useful forecast prediction intervals. Lets call the new function calculate_intervals(). It'll take a dataset of forecasts--1 or more time series--a dataset of historical predictions and actuals, and some additional arguments for importance weighting. The importance weighting will make it easy to change forecast intervals on the fly if, say, the world is moving into a more uncertain regime like all of 2020 and you want to mimic this uncertainty by sampling residuals in the most 2020-esque time period in the past.

It'll look roughly like forecastML::calculate_intervals(forecasts, predictions, weights = NULL, ...)

I'm not sure what the return value will look like at the moment.

[edgBR]

Hi Nick,

To be fully complaint with any ml technique like for instance quantile forest regression, would it be possible to select what you are using to get the prediction intervals like for instance:

forecastML::calculate_intervals(forecasts, predictions, , from="bootstrap_residuals", weights = NULL, ...)

forecastML::calculate_intervals(forecasts, predictions, , from="quantile_loss", weights = NULL, ...)

BR /Edgar

[nredell]

Good timing. I was actually messing around with the quantregForest package right now. We'll see. I'm trying to shift things into an even more generic direction ala the new reconcile_forecasts() function where things work with forecastML objects but also work generically. TBD. Thanks for the input.

[nredell]

calculate_intervals() added. Not tested on grouped data with multiple time series yet.

[edgBR]

Hi Nick,

I was looking into the last commits and I noticed that you are using:

residuals <- function (x, ...) {
  UseMethod("residuals", x)
}

To get the residuals to perform bootstrapping. Is this method available for the model objects coming from python?

BR /Edgar

[nredell]

The desired behavior is now up and running. Works with multiple models and groups now. Check out the end of the updated python notebook for a worked example. I'm still working through the generic part of calculate_intervals() that would take in custom residuals as a data.frame.