How to set input_size

[StatMixedML]

Description

What does the input_size argument in DeepAREstimator or TransformerTempFlowEstimator stand for and how to properly set meaningful values for each of them? Would it be possible to derive the values directly from the input data?

[kashif]

@StatMixedML yes for now I need to set that manually, i set it to zero and check the error to see what it "should" be... its the size of the final feature which the LSTM/GRU or Transformer's input layer expects... and depends on the freq of the dataset the lags that get added and other embeddings that get concated etc.

I need to come up with a helper function which will get rid of the need of input size... but havent gotten around to it. For now just start with input_size=1 and then set it to what the error message says it should be...

[StatMixedML]

@kashif Thanks for clarifying. It works for DeepAREstimator, but when using input_size=1 in TransformerTempFlowEstimator

torch.manual_seed(123)
trainer = Trainer(epochs = 10) 

estimator = TransformerTempFlowEstimator(input_size = 1,
                                         freq = "1M", 
                                         prediction_length = 12,
                                         target_dim = 133,
                                         trainer = trainer,
                                         cardinality = [7, 20])                              
predictor = estimator.train(training_data = train_ds)

it throws the following error

0it [00:00, ?it/s]
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-46-4ee0f952b967> in <module>
     10                                          trainer = trainer,
     11                                          cardinality = card_static)                              
---> 12 predictor = estimator.train(training_data = train_ds)

~/miniconda3/envs/pytorchts/lib/python3.7/site-packages/pts/model/estimator.py in train(self, training_data)
    133 
    134     def train(self, training_data: Dataset) -> Predictor:
--> 135         return self.train_model(training_data).predictor

~/miniconda3/envs/pytorchts/lib/python3.7/site-packages/pts/model/estimator.py in train_model(self, training_data)
    121             net=trained_net,
    122             input_names=get_module_forward_input_names(trained_net),
--> 123             data_loader=training_data_loader,
    124         )
    125 

~/miniconda3/envs/pytorchts/lib/python3.7/site-packages/pts/trainer.py in __call__(self, net, input_names, data_loader)
     50                     inputs = [data_entry[k].to(self.device) for k in input_names]
     51 
---> 52                     output = net(*inputs)
     53                     if isinstance(output, (list, tuple)):
     54                         loss = output[0]

~/miniconda3/envs/pytorchts/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

~/miniconda3/envs/pytorchts/lib/python3.7/site-packages/pts/model/transformer_tempflow/transformer_tempflow_network.py in forward(self, target_dimension_indicator, past_time_feat, past_target_cdf, past_observed_values, past_is_pad, future_time_feat, future_target_cdf, future_observed_values)
    352 
    353         enc_out = self.transformer.encoder(
--> 354             self.encoder_input(enc_inputs).permute(1, 0, 2)
    355         )
    356 

~/miniconda3/envs/pytorchts/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

~/miniconda3/envs/pytorchts/lib/python3.7/site-packages/torch/nn/modules/linear.py in forward(self, input)
     85 
     86     def forward(self, input):
---> 87         return F.linear(input, self.weight, self.bias)
     88 
     89     def extra_repr(self):

~/miniconda3/envs/pytorchts/lib/python3.7/site-packages/torch/nn/functional.py in linear(input, weight, bias)
   1370         ret = torch.addmm(bias, input, weight.t())
   1371     else:
-> 1372         output = input.matmul(weight.t())
   1373         if bias is not None:
   1374             output += bias

RuntimeError: size mismatch, m1: [768 x 401], m2: [1 x 32] at /tmp/pip-req-build-7mav6f4d/aten/src/TH/generic/THTensorMath.cpp:197

How do I properly set the values for input_size here? Many thanks.

[kashif]

yes here set it to 401

[StatMixedML]

Great, working!