Working Example of TransformerTempFlowEstimator

[AndreRoehrig]

I have been trying to get TransformerTempFlowEstimator working without success. Can you provide an example script? Issues include RuntimeError: Sizes of tensors must match except in dimension 2. Got 1 and 32 in dimension 0 and not understanding how the data loading works for multivariate data. My example below:

from pts.dataset import MultivariateGrouper
import pandas as pd
import torch

from pts.dataset import ListDataset
from pts.model.transformer_tempflow import TransformerTempFlowEstimator
from pts import Trainer

url = "https://raw.githubusercontent.com/numenta/NAB/master/data/realTweets/Twitter_volume_AMZN.csv"
df = pd.read_csv(url, header=0, index_col=0, parse_dates=True)

train_ds = ListDataset(
    [{"start": df.index[0], "target": df.value[:"2015-04-05 00:00:00"]+i}
        for i in range(2)],
    freq="5min"
)

grouper_train = MultivariateGrouper(max_target_dim=2)
gt = grouper_train(train_ds)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

trainer = Trainer(epochs=10)

estimator = TransformerTempFlowEstimator(input_size=1,
                                         freq="5min",
                                         prediction_length=100,
                                         context_length=4,
                                         target_dim=64,
                                         cardinality=[7, 20],
                                         trainer=trainer)

predictor = estimator.train(training_data=gt)

[kashif]

@AndreRoehrig thanks for the question. The example from the README is just a single time series whereas the TransformerTempFlowEstimator is a multivariate model, i.e. it expects more than 1 time series in parallel. I can post an example with another dataset here shortly.

[AndreRoehrig]

@kashif I realise that the example is just for a single time series. Hence, I tried to adapt it with MultivariateGrouper. I'm also happy to see an example with another data set.

[Sanyorke-Han]

I run above sample code, encounted exception below. also with the sample code in the readme doc, got the similar exceptions:

RuntimeError Traceback (most recent call last)

in 20 trainer=trainer) 21 ---> 22 predictor = estimator.train(training_data=gt) C:\Python37\lib\site-packages\pts\model\estimator.py in train(self, training_data) 146 147 def train(self, training_data: Dataset) -> Predictor: --> 148 return self.train_model(training_data).predictor C:\Python37\lib\site-packages\pts\model\estimator.py in train_model(self, training_data) 134 net=trained_net, 135 input_names=get_module_forward_input_names(trained_net), --> 136 data_loader=training_data_loader, 137 ) 138 C:\Python37\lib\site-packages\pts\trainer.py in __call__(self, net, input_names, data_loader) 50 inputs = [data_entry[k].to(self.device) for k in input_names] 51 ---> 52 output = net(*inputs) 53 if isinstance(output, (list, tuple)): 54 loss = output[0] C:\Python37\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs) 548 result = self._slow_forward(*input, **kwargs) 549 else: --> 550 result = self.forward(*input, **kwargs) 551 for hook in self._forward_hooks.values(): 552 hook_result = hook(self, input, result) C:\Python37\lib\site-packages\pts\model\transformer_tempflow\transformer_tempflow_network.py in forward(self, target_dimension_indicator, past_time_feat, past_target_cdf, past_observed_values, past_is_pad, future_time_feat, future_target_cdf, future_observed_values) 343 future_time_feat=future_time_feat, 344 future_target_cdf=future_target_cdf, --> 345 target_dimension_indicator=target_dimension_indicator, 346 ) 347 C:\Python37\lib\site-packages\pts\model\transformer_tempflow\transformer_tempflow_network.py in create_network_input(self, past_time_feat, past_target_cdf, past_observed_values, past_is_pad, future_time_feat, future_target_cdf, target_dimension_indicator) 239 240 # (batch_size, target_dim, embed_dim) --> 241 index_embeddings = self.embed(target_dimension_indicator) 242 # assert_shape(index_embeddings, (-1, self.target_dim, self.embed_dim)) 243 C:\Python37\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs) 548 result = self._slow_forward(*input, **kwargs) 549 else: --> 550 result = self.forward(*input, **kwargs) 551 for hook in self._forward_hooks.values(): 552 hook_result = hook(self, input, result) C:\Python37\lib\site-packages\torch\nn\modules\sparse.py in forward(self, input) 112 return F.embedding( 113 input, self.weight, self.padding_idx, self.max_norm, --> 114 self.norm_type, self.scale_grad_by_freq, self.sparse) 115 116 def extra_repr(self): C:\Python37\lib\site-packages\torch\nn\functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse) 1722 # remove once script supports set_grad_enabled 1723 _no_grad_embedding_renorm_(weight, input, max_norm, norm_type) -> 1724 return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse) 1725 1726 RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.IntTensor instead (while checking arguments for embedding) -----------------------------------------

[kashif]

fixed by 6749293