sktime / sktime

A unified framework for machine learning with time series
https://www.sktime.net
BSD 3-Clause "New" or "Revised" License
7.56k stars 1.29k forks source link

[ENH] Extend forecasting benchmark to global setting #6661

Open benHeid opened 6 days ago

benHeid commented 6 days ago

As far as I know, currently the forecasting benchmarking (as well as evaluate) is not working for global setting. I.e., fitting a forecasting model on time series 1:n and evaluating it on n+1:m with n<m.

@fkiraly please mention if it is already implemented.

fkiraly commented 6 days ago

No, not implemented. For this, evaluate would have to be adapted I think.

I would be surprised if it worked out of the box - it could be in theory because there is an InstanceSplitter which splits a collection of time series by instance, so it might be if we pass that it works. Even if, this is perhaps not the best user experience.

Possible improvements: