TFT Issue with Multiple Static Categorical Features

[esbraun]

When passing 2 or more static categorical features to the TFT implementation using a PandasDataset, the following errors occur:

File ~/tft/lightning_module.py:76, in TFTLightningModule.forward(self, batch) 59 future_observed_values = batch["future_observed_values"] 61 ( 62 tft_target, 63 time_feat, (...) 74 future_target=future_target, 75 ) ---> 76 params = self.model.output_params( 77 tft_target, 78 time_feat=time_feat, 79 embedded_cat=embedded_cat, 80 static_feat=static_feat, 81 ) 82 distr = self.model.output_distribution(params, scale) 84 loss_values = self.loss(distr, future_target)

File ~/tft/module.py:503, in TFTModel.output_params(self, target, time_feat, embedded_cat, static_feat) 499 future_time_feat_proj = time_feat_proj[:, self.context_length :, ...] 501 static_feat_proj = self.static_feat_proj(static_feat) --> 503 staticvar, = self.static_selection(embedded_cat + [static_feat_proj]) 504 static_selection = self.selection(static_var).unsqueeze(1) 505 static_enrichment = self.enrichment(static_var).unsqueeze(1)

TypeError: can only concatenate tuple (not "list") to tuple

[kashif]

i see... let me see if i can fix it

[esbraun]

test case.zip I updated the sample notebook to illustrate a test case.

[kashif]

thank you! i will test

[esbraun]

Any intuition on an approach to fix the error? I want to take a hand at this, but a little guidance would go a long way.

[kashif]

ah damn! so sorry @esbraun I lost track of this... hmm I was working on paper deadlines.... when do you need it?

[esbraun]

I couldn't in good conscience give you a deadline considering this is a pro-bono effort! I'm already using this implementation now with a hacky approach for turning categorical features into continuous features; I'm seeing much better results for my application than the gluonts implementation. I would greatly appreciate a fix when you do have time.

[kashif]

no worries! can you ping me on Wednesday to remind me?

[esbraun]

Sure, and thank you very kindly. I do think the gluonts user community would also greatly benefit from this implementation!

[kashif]

ok cool once we fix up your issue i will get it merged into gluonts. Thanks!

[esbraun]

If merging the model is a near term goal, I did also find an issue as well with "past_feat_dynamic_real" features, though not when they are lags defined by the TFTEstimator parameter 'lags_seq'. I'll make a reproducible example and post it.

[kashif]

yes please!

[esbraun]

I tested past dynamic features using the stallion dataset from https://github.com/jdb78/pytorch-forecasting and wasn't able to reproduce the issue. My current take is the past dynamic feature issue I saw wasn't related to TFTEstimator.

[esbraun]

@kashif I see you've implemented a number of updates - thank you for iterating the repo. Have you been able to address the static categorical feature issue with TFT?