Add support for exogenous regressors

Background

This pull request begins the process of adding support for exogenous regressors in Merlion. Consider a multivariate time series $X^{(1)}, \ldots, X^{(t)}$, where each $X^{(i)} \in \mathbb{R}^d$ is a d-dimensional vector. In multivariate forecasting, our goal is to predict the future values of the k'th univariate $X_k^{(t+1)}, \ldots, X_k^{(t+h)}$.

Exogenous regressors $Y^{(i)}$ are a set of additional variables whose values we know a priori. The task of forecasting with exogenous regressors is to predict our target univariate $X_k^{(t+1)}, \ldots, X_k^{(t+h)}$, conditioned on

The past values of the time series $X^{(1)}, \ldots, X^{(t)}$
The past values of the exogenous regressors $Y^{(1)}, \ldots, Y^{(t)}$
The future values of the exogenous regressors $Y^{(t+1)}, \ldots, Y^{(t+h)}$

For example, one can consider the task of predicting the sales of a specific item at a store. Endogenous variables $X^{(i)} \in \mathbb{R}^4$ may contain the number of units sold (the target univariate), the consumer price index, the temperature outside, and whether a competing store is open. Exogenous variables $Y^{(i)} \in \mathbb{R}^3$ are variables that the store has control over, and they may include whether a particular day is a holiday, the price the store is selling the item for, and the percentage discount being offered on the item.

A more detailed tutorial can be found here.

Implementation Details

This pull request refactors the base forecaster classes to accommodate exogenous regressors at both training and inference time. Now, the exog_data argument can be passed to the train() or forecast() method of any non-autoML forecaster, ensembles of forecasters, or forecast evaluators. We also allow exog_data to be passed to the train(), forecast() and get_anomaly_score() methods of forecast-based anomaly detectors. If a particular model doesn't implement support for exogenous regressors but exogenous regressors are provided, the model simply emits a warning and falls back to ordinary forecasting behavior. Models which explicitly support exogenous regressors must inherit from the new ForecasterExogBase instead of ForecasterBase.

Behind the scenes, when exog_data is passed to model.train(), we apply an optional exog_transform to the exogenous data and then resample it to the same timestamps as the train data (this is using the exog_aggregation_policy and exog_missing_value_policy specified in the config). We call the implementation-specific _train_with_exog() method if exog_data is given, or the regular _train() if it is not. When forecast() is called, we first ensure that the exog_data has the same dimension as what was given to train(), apply any desired transform or resampling, and then call either _forecast_with_exog() or _forecast().

Besides adding API support at the level of the base class, we enhance the SARIMA and Prophet models to support exogenous regressors. Since the models are already capable of supporting exogenous regressors, this change amounts to simply modifying the underlying library calls. We also refactor layered models and ensembles to use the internal _train() and _train_with_exog() APIs (previously they directly implemented a train() method). This allows us to easily use layered models (including AutoML models) and ensembles in conjunction with exogenous regressors.

Other Changes

This PR forces inverse transforms to rely exclusively on named univariates (rather than indices). Before this change, incorrect behavior would occur when attempting to apply the inverse transform on the forecast (univariate) produced by a multivariate forecasting model. Additionally, we add a new property identity_inversion to each transform, which indicates whether their inverse transform is just the identity. If the inverse transform is just the identity, we can skip applying it altogether to speed up the process of post-processing a model's forecast.

Additionally, this PR introduces a GridSearch object for hyperparameter search, and we use it for AutoProphet and AutoETS. We anticipate that objects encapsulating the logic of hyperparameter search will be useful in parallelizing hyperparameter optimization in the future.

Finally, we deprecate support for Python 3.6 and update the version to 1.3.1.

Limitations and Future Work

For the time being, exogenous regressors are only supported for SARIMA and Prophet. As the feature matures, we intend to implement exogenous regressor support for deep models (once the implementations become more mature) and tree-based autoregressive models. We also need to add support for exogenous regressors to the Spark API.

salesforce / Merlion