Infomer with different output_size

[antonio-navarra]

Feature request

The Infomer transformers seems to perform well on time series multivariate forecasting also in my application, however, it would be nice to have the option to have an output_size for the features different from the input_size.

Motivation

The reasons is that for the multivariate forecasting problem i am considering it would make sense to focus the predictions only on the variables/features of interest.

Your contribution

I will be happy to work and contribute what I can. I have inspected the code -- I am using InfomerforPrediction -- but it i snot immediately obvious to me where to insert the last fully connected layer for the reduction. If somebody can give some guidance, i will be happy to try it...

[amyeroberts]

cc @kashif

[kashif]

@antonio-navarra yes so my intial setup was to have any extra dynamic covariates (which will be part of the input vector) as dynamic real features (with the caviate that they are known at inference time). For covariates that are only know in the context window, then yes one would need some extra features....

what was your setup and can it be done via dynamic real features?

[antonio-navarra]

Kashif,

my setup is that I have a number of physical fields, e.g Field1, Field2, etc.., each one is described by a different number of feature and relative time series. Say, to fix the ideas that I have Field1 (6 features), Field2 (11) and so on.. The length of the time series is the same for all of them.

The target is to predict the Fields. In the present formulation , the target is the same number of features over some prediction time

Train (Field1, Field2) —> Predict (Field1, Field2)

I had some successes, but I thought that it might be interesting to try

Train (Field1, Field2) —> Predict (Field1)

The extra feature cannot be used as dynamical real features because they are not known at prediction time…

That’s the situation, I am glad for any help…

Best

Antonio

On 2 Nov 2023, at 17:08, Kashif Rasul @.***> wrote:

@antonio-navarra https://github.com/antonio-navarra yes so my intial setup was to have any extra dynamic covariates (which will be part of the input vector) as dynamic real features (with the caviate that they are known at inference time). For covariates that are only know in the context window, then yes one would need some extra features....

what was your setup and can it be done via dynamic real features?

— Reply to this email directly, view it on GitHub https://github.com/huggingface/transformers/issues/27238#issuecomment-1791031332, or unsubscribe https://github.com/notifications/unsubscribe-auth/APQWRZ74DOFOWM77XPQE7O3YCPAOJAVCNFSM6AAAAAA62V6TVGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJRGAZTCMZTGI. You are receiving this because you were mentioned.

[firmai]

The time series dataset processing has to be improved. Currently it asks for target, what if what we care about it multi-target (i.e., multivariate series), even the new PatchTSMixer is entirely unusable, no idea how to get the data formatted. Went through all your helper notebooks, not helpful at all. For example for PatchTSMixer it would be extremely helful to have an example that goes from: numpy_data # Reshaped to (num_samples, lookback_window, num_features)

to trainer = Trainer(model=model, train_dataset=train_scaled, eval_dataset=valid_scaled)

[firmai]

This is the current notebook, I think a real explanatory notebook would very quickly increase the uptake for TS hugging. https://github.com/huggingface/notebooks/blob/main/examples/time_series_datasets.ipynb

[kashif]

@firmai so the informer example is a multivariate setup, that takes as input the multivariate vector and on the output side the emission head is an independent diagonal. The patchtstmixer is multivariate and a blog post https://github.com/huggingface/blog/pull/1740 with a notebook is in the review stages. Could you kindly have a look?

[firmai]

Perfect, I knew there was something missing, I guess I just jumped on the train to early. Thanks.