How future values are identified in the Dataset?

[polal2is]

Hi, Being used to custom tensorflow models for time series forecasting, I got a bit puzzled about how known future data are handled here. I usually use dataframe shifts applied to these know features (like weather forecast for instance).

How does the TFT model knows where future (known) values start from the TimeSeriesDataSet()? Examining the getstallion data set didn't help as all time series have values for every feature up to end of 2017 would have expected NaNs for unknown features at a given date and/or a parameter to set a "now" time index in the dataframe.

I think it would be great to comment a bit more on this in the TFT tutorial, I got confused despite having experience with time series so maybe it will help others as well.

Many thanks in advance

[jdb78]

Thanks for motivating an extension of the TFT tutorial!

You can shift things only for Autoregressive models and handling the shifts if less than the forecasting horizon is important. For other models like transformers, data should not be shifted. This is why things are being handled through the TimeSeriesDataSet in which you define the variables that are unkown in the future. For the TFT, for example, you just ignore these variables in the decoding stage.

[polal2is]

Thanks for your reply! Maybe I should have been more specific.

For a single time series with 2 features (a known and an unknown), a real world dataset will typically look like this:

Is this a proper formatting to feed into 'predict' for new prediction on most current data?

Hope to be clear :) Thanks

[polal2is]

I think I got it by having a look at 'encoder_data' and 'decoder_data' of the 'Predict on new data' section of TFT tutorial.

Your statement 'For the TFT, for example, you just ignore these variables in the decoding stage' makes more sense now.

That should work for the above dataset I guess then:

And unknown reals will be ignored