jdb78
commented
3 years ago Thanks for starting this discussion!
Combine neural nets with statistical models
Feel free to implement an approach - maybe even univariate statistical models - PRs are always welcome! To my knowledge ES-RNN (the one who won the M4 competition) requires different architectures for different time horizons and has been beaten by the far more generic N-BEATS network which is also implemented in PyTorch Forecasting. Prophet seems to perform well for the application it is designed for (isolated interventions in otherwise stable time series) but does not perform well in other benchmarks even compared to very basic forecasting techniques such as the theta method. It has also only very limited incorporation of covariates which is an issue for real world applications. I am not sure what exactly neural prophet adds to improve it.
Transfer learning
If you know of a reliable and performant way to do it, I am very keen to read the article and implement something in PyTorch Forecasting! To my knowledge attempts to implement transfer learning in the domain of time series have largely not been very successful so far. The biggest issue seems to be that it is such an heterogeneous task.
Combine neural nets with statistical models Some papers/competitions have shown that combining statistical models with neural networks can increase accuracy. An example is the winner of the M4-competition.
An example beta project with a similar approach is this one: https://github.com/ourownstory/neural_prophet
A potential benefit that I see with this kind of approach is that the model can rely more/less on the statistical model based on the amount of data available. If you train on 100,000 different time series the model would primarily rely on the neural network. If you train on 5 time series the model could rely primarily on the statistical model. This way we could have good performance on any forecasting problem using only pytorch-forecasting instead of having to switch between frameworks and create individual data structures and features for each of them.
Transfer learning In other domains, primarily image recognition and NLP, transfer learning has become a standard. Having pre-trained time series models that we can use for transfer learning would be fantastic. I imagine the pre-trained model would be trained on endogenous target data and for finetuning you would adjust for your own target data and also add any extra covariates and embeddings for categoricals that you have.
Are there any plans or interest regarding implementing something like this in pytorch-forecasting?