What happens when i fit a multivariate time series?

[alejandrogomez97]

I'm fitting a tft on my series, which is a time series object with 11 variables/components. I'm interested in predicting the 11 components of my time series, but I'm not sure of what's happenning when i fit a model on my series like this. To predict a single component is it using other components as past covariates automatically? Can this be done? Is it creating a global model to predict the 11 components of my time series or the model is different for every component?

This is the code i'm using:

df=pd.read_parquet('datosArreglados.parquet')
df=df.reset_index(drop=True)
puntos=30000
inicio=0
train=df[inicio:-puntos]
test=df[-puntos:]
test=test[:29016]
train = TimeSeries.from_dataframe(train)
test= TimeSeries.from_dataframe(test)
series= TimeSeries.from_dataframe(df)
transformer = Scaler()
train_transformed = transformer.fit_transform(train)
val_transformed = transformer.transform(test)
series_transformed = transformer.transform(series)
# default quantiles for QuantileRegression
quantiles = [
    0.01,
    0.05,
    0.1,
    0.15,
    0.2,
    0.25,
    0.3,
    0.4,
    0.5,
    0.6,
    0.7,
    0.75,
    0.8,
    0.85,
    0.9,
    0.95,
    0.99,
]
input_chunk_length = 200
forecast_horizon = 2
my_model = TFTModel(
    input_chunk_length=input_chunk_length,
    output_chunk_length=forecast_horizon,
    hidden_size=64,
    lstm_layers=2,
    num_attention_heads=4,
    dropout=0.2,
    batch_size=64,
    n_epochs=200,
    add_relative_index=True,
    add_encoders=None,
    likelihood=QuantileRegression(
        quantiles=quantiles
    ),  # QuantileRegression is set per default
    # loss_fn=MSELoss(),
    random_state=42,
)
my_model.fit(train_transformed, verbose=True)

Thanks in advance.

[madtoinou]

Hi,

The way the components of a multivariate TimeSeries are "used" ultimately depends on the model that you're using. Some models have modules specifically designed to support them: the TFTModel that you're using for example, explicitly tries to select relevant features and filter out the others according to the original article whereas other model such as NBeats reshape the input to transform it into an univariate series since the model only support this kind of input (forward method of _NBEATSModule in the source code).

If you want the model to consider some components as past/future covariates, it's better to explicitly use them as such because again, depending on the model, a specialized module might take care of them differently than the rest.

I hope that my answer solves your issue. Since we had several question about this, we might expand one of the example to better explain this aspect of the models behavior.

[dwolffram]

Hi @madtoinou , I currently have the same questions and want to ask for some clarification. What exactly is the difference in darts if I train a TFTModel on a multivariate series vs. on multiple univariate series? From my understanding of the original paper, TFT is meant for multiple entities with univariate targets, so it's trained in a global fashion. Is the multivariate case then just transformed into multiple series under the hood, or how does it work? Are all components then automatically used as covariates or not? I'm a bit confused because first you kind of said that TFT can handle this and then you said it's better to include all components as covariates (or was that meant for other models?). Sorry, if I misunderstood something, but understanding the details of this is important for my current project. Thanks!

[madtoinou]

Hi @dwolffram,

The key point is that covariates are not forecasted. Hence, using the multivariate approach is interesting when you are actually interested in forecasting all its components. If some of them don't need to be forecasted, it's better to use them as covariates as some model treat them in a slightly different way.

Nevertheless, when you pass a multivariate series to the model, it will have access to all the components "as features" to forecast each component (hence capture relationship between them). It's true that the wording "reshape the input to transform it into an univariate series" can bring confusion, I meant to say "reshape & flatten into a one dimensional tensor" when the model architecture does not support 2D tensors.

[dwolffram]

Hi @madtoinou (or @dennisbader),

thanks for your answer! Yes, I want to forecast all components. Just to clarify:

• With "as features" do you mean that they are treated the same as past_covariates?
• From my understanding, TFT is a global univariate model, so it is not designed for multivariate targets (see also here for example). Does that mean one model per component is trained in darts? Or does TFTModel internally treat a multivariate target as multiple series and train one global model? (What's the difference with training on multiple univariate series then?)

[madtoinou]

Hello again @dwolffram,

• Target features are not necessarily processed the same way as past_covariates, it depends on the model architecture. But if there are several target components, they will all be processed the same way ("as target"). In the context of TFTModel, they are not processed in the same way as the mandatory future_covariates which go through specific modules for example.
• The original article indeed introduces TFT as an univariate model, but since it allows multi-horizon (hence generate several values from a given input), we can train it to forecast several horizons across several components. You can see the reshaping of the output here and still benefits from the model overall architecture.

You could compare the performance of a TFT model trained on the multivariate series and several of them, each one of them trained on one of the target component but Darts is definitely not doing it under the hood for the torch-based models.

You can see a clear explanation of the difference between multivariate and multiple series here, as far I know, there is no scenario in which Darts does this paradigm shift automatically (or at all, to be honest).