modeltime_resample_accuracy with 1 observation test sets

[csanda14]

Hi @mdancho84,

I cannot for the life of me get the function modeltime_resample_accuracy to work when using 1 observation in each test set.

When running:

resamples = time_series_cv(
  data = score_df,
  initial = "180 days",
  assess = "1 days"
)

... models ... 

resamples_fitted = model_table %>% 
  modeltime_fit_resamples(
    resamples = resamples,
    control = control_resamples(verbose = FALSE)
  )

resamples_fitted %>%
  modeltime_resample_accuracy(summary_fns = mean) %>%
  table_modeltime_accuracy(.interactive = FALSE)

I get the following error:

Error: In metric: `mase`
Problem with `summarise()` column `.estimate`.
i `.estimate = metric_fn(...)`.
x `truth` must have a length greater than `m` to compute the out-of-sample naive mean absolute error.
i The error occurred in group 1: .model_id = 1, .model_desc = "ARIMA(2,0,2)(1,1,2)[7] WITH DRIFT", .resample_id = "Slice01", .type = "Resamples".

Using 2 days instantly fixes the problem, but this is not what I want. I want to judge the average day-ahead performance of each model, e.g. over 30 days.

[mdancho84]

This is because mase fails. It can be removed. We use default_forecast_accuracy_metric_set(), which contains mase. You can create a new metric_set() without mase.

[csanda14]

Thank you! It works now. Should have been able to figure that out myself. Oh well, sometimes you are just blind!