Steviey
commented
2 years ago +1
Open vidarsumo opened 3 years ago
Steviey
commented
2 years ago +1
mdancho84
commented
1 year ago For those who stumble up on this question. Then here is a sample code showing how you can do this with a for loop.
library(modeltime) library(modeltime.ensemble) library(timetk) library(tidymodels)
data_tbl <- walmart_sales_weekly %>% filter(id == "1_3") %>% select(Date, Weekly_Sales) %>% set_names("date", "value")
splits <- data_tbl %>% time_series_cv(date_var = date, assess = 12, skip = 1, cumulative = TRUE, slice_limit = 5)
initial_fit_data_tbl <- splits$splits[[nrow(splits)]] %>% training()
arima_mtbl <- arima_reg() %>% set_engine("auto_arima") %>% fit(value ~ date, data = initial_fit_data_tbl) %>% modeltime_table()
ets_mtbl <- exp_smoothing() %>% set_engine("ets") %>% fit(value ~ date, data = initial_fit_data_tbl) %>% modeltime_table()
models_mtbl <- combine_modeltime_tables( arima_mtbl, ets_mtbl )
models_mtbl <- models_mtbl %>% ensemble_average() %>% modeltime_table() %>% combine_modeltime_tables(models_mtbl)
models_mtbl %>% modeltime_fit_resamples(splits) %>% modeltime_resample_accuracy()
accuracy_ls <- list() # to store the accuracy calculations in
for(i in 1:nrow(splits)) {
print(glue::glue("Resample: {i}"))
# Create train and validation/test data sets
train_data_loop_tbl <- splits$splits[[i]] %>% training()
test_data_loop_tbl <- splits$splits[[i]] %>% testing()
# Refit model and calculate accuracy
accuracy_tbl <- models_mtbl %>%
modeltime_refit(train_data_loop_tbl) %>%
modeltime_accuracy(new_data = test_data_loop_tbl) %>%
# just so I know which resample this accuracy calculations refers too
# This is not important. Just informative
mutate(resample = paste0("resample_", i))
accuracy_ls[[i]] <- accuracy_tbl}
accuracy_tbl <- accuracy_ls %>% bind_rows()
accuracy_tbl %>% group_by(.model_desc) %>% summarise( mean_rmse = mean(rmse), mean_mae = mean(mae) ) %>% arrange(mean_rmse)
This code will run but the ensemble is not present when I calculate accuracy:
I.e. the output of modeltime_resample_accuracy() will only include the models, not the ensemble.
Same if I have a table with stacked ensembles and try to run this:
Session info: