Increasing Forecasting Horizon without affecting training window in backtesting_forecaster function

[udishadc]

I'm using the backtesting_forecaster function in skforecast for time series forecasting.

Currently, the steps parameter controls both the forecasting horizon and the number of steps by which training window is moving in each iteration. Is there a way to independently adjust the forecasting horizon without affecting the training window?

Code Snippet:

steps = 24
metric, predictions_backtest = backtesting_forecaster(
                                   forecaster         = forecaster,
                                    y                  = data_train["y"][:24*100],                                
                                   initial_train_size = 24*10,
                                   fixed_train_size   = True,
                                   steps              = steps,
                                   metric             = 'mean_squared_error',
                                   refit              = True,
                                   verbose            = True,
                                   show_progress      = True

)
print(f"Backtest metric (MSE): {metric}") 

Output:

Information of backtesting process
----------------------------------
Number of observations used for initial training: 240
Number of observations used for backtesting: 2160
    Number of folds: 90
    Number of steps per fold: 24
    Number of steps to exclude from the end of each train set before test (gap): 0

Fold: 0
    Training:   2006-01-01 00:00:00 -- 2006-01-10 23:00:00  (n=240)
    Validation: 2006-01-11 00:00:00 -- 2006-01-11 23:00:00  (n=24)
Fold: 1
    Training:   2006-01-02 00:00:00 -- 2006-01-11 23:00:00  (n=240)
    Validation: 2006-01-12 00:00:00 -- 2006-01-12 23:00:00  (n=24)
Fold: 2
    Training:   2006-01-03 00:00:00 -- 2006-01-12 23:00:00  (n=240)
    Validation: 2006-01-13 00:00:00 -- 2006-01-13 23:00:00  (n=24)
Fold: 3
    Training:   2006-01-04 00:00:00 -- 2006-01-13 23:00:00  (n=240)
    Validation: 2006-01-14 00:00:00 -- 2006-01-14 23:00:00  (n=24)
Fold: 4
    Training:   2006-01-05 00:00:00 -- 2006-01-14 23:00:00  (n=240)
    Validation: 2006-01-15 00:00:00 -- 2006-01-15 23:00:00  (n=24)
Fold: 5
    Training:   2006-01-06 00:00:00 -- 2006-01-15 23:00:00  (n=240)
...
Fold: 89
    Training:   2006-03-31 00:00:00 -- 2006-04-09 23:00:00  (n=240)
    Validation: 2006-04-10 00:00:00 -- 2006-04-10 23:00:00  (n=24)

[JoaquinAmatRodrigo]

Hi @udishadc, Thanks for using skforecast. Currently, the training window is tied to the forecast horizon (steps). Three different configurations are allowed:

• refit = False : only the initial training is applied.
• refit = True : the training window moves steps positions in each fold
• refit = <integer> : the training is repeated each <integer> folds.

The reason for this behavior is that the current skforecast API only allows 1 predicted value per step. What kind of behavior are you looking for? Your feedback may help us include new features in future releases.

[udishadc]

Hi @JoaquinAmatRodrigo, Thank you for the clarification. I was looking to evaluate model performance across different forecasting horizons while keeping the training window constant. I understand the current limitations of the steps parameter and the reasons behind it. Thank you.