TypeError: PyTorchPredictor.__init__() got an unexpected keyword argument 'freq'

[simonyelisey]

Hi folks!

I'm trying to reproduce Time-Grad example notebook and getting this error:

TypeError                                 Traceback (most recent call last)
Cell In[50], line 1
----> 1 predictor = estimator.train(electricity_dataset_train, prefetch_factor=None)

File ~/Documents/Projects/00_GenModels/notebooks/../pytorch-ts/pts/model/estimator.py:179, in PyTorchEstimator.train(self, training_data, validation_data, num_workers, prefetch_factor, shuffle_buffer_length, cache_data, **kwargs)
    169 def train(
    170     self,
    171     training_data: Dataset,
   (...)
    177     **kwargs,
    178 ) -> PyTorchPredictor:
--> 179     return self.train_model(
    180         training_data,
    181         validation_data,
    182         num_workers=num_workers,
    183         prefetch_factor=prefetch_factor,
    184         shuffle_buffer_length=shuffle_buffer_length,
    185         cache_data=cache_data,
    186         **kwargs,
    187     ).predictor

File ~/Documents/Projects/00_GenModels/notebooks/../pytorch-ts/pts/model/estimator.py:160, in PyTorchEstimator.train_model(self, training_data, validation_data, num_workers, prefetch_factor, shuffle_buffer_length, cache_data, **kwargs)
    141     validation_data_loader = DataLoader(
    142         validation_iter_dataset,
    143         batch_size=self.trainer.batch_size,
   (...)
    148         **kwargs,
    149     )
    151 self.trainer(
    152     net=trained_net,
    153     train_iter=training_data_loader,
    154     validation_iter=validation_data_loader,
    155 )
    157 return TrainOutput(
    158     transformation=transformation,
    159     trained_net=trained_net,
--> 160     predictor=self.create_predictor(
    161         transformation, trained_net, self.trainer.device
    162     ),
    163 )

File ~/Documents/Projects/00_GenModels/notebooks/../pytorch-ts/pts/model/time_grad/time_grad_estimator.py:249, in TimeGradEstimator.create_predictor(self, transformation, trained_network, device)
    246 input_names = get_module_forward_input_names(prediction_network)
    247 prediction_splitter = self.create_instance_splitter("test")
--> 249 return PyTorchPredictor(
    250     input_transform=transformation + prediction_splitter,
    251     input_names=input_names,
    252     prediction_net=prediction_network,
    253     batch_size=self.trainer.batch_size,
    254     freq=self.freq,
    255     prediction_length=self.prediction_length,
    256     device=device,
    257 )

TypeError: PyTorchPredictor.__init__() got an unexpected keyword argument 'freq'

Does anybody know solution?

Dependencies:

python==3.11.4
gluonts==0.10.0
pytorchts - master branch

[kashif]

please use the 0.7.0 branch and try?

[simonyelisey]

@kashif

I cloned 0.7.0 branch but it has caused new error:

ImportError                               Traceback (most recent call last)
Cell In[5], line 5
      3 from pts.model.time_grad import TimeGradEstimator
      4 # from pts.model.transformer_tempflow import TransformerTempFlowEstimator
----> 5 from pts import Trainer
      7 from diffusers import DEISMultistepScheduler

ImportError: cannot import name 'Trainer' from 'pts' (.../pytorch-ts/pts/__init__.py)

[kashif]

hmm that still looks like its the older stuff...

[simonyelisey]

I solved ImportError from previous comment by adding trainer.py to pts from master to 0.7.0 and editing init.py in the same folder.

But I've got new error seems like #158

ValidationError                           Traceback (most recent call last)
Input In [22], in <cell line: 3>()
      1 # scheduler = DEISMultistepScheduler(num_train_timesteps=150, beta_end=0.1)
----> 3 estimator = TimeGradEstimator(
      4     target_dim=int(electricity.metadata.feat_static_cat[0].cardinality),
      5     prediction_length=electricity.metadata.prediction_length,
      6     context_length=electricity.metadata.prediction_length,
      7     cell_type='GRU',
      8     input_size=1288,
      9     freq=electricity.metadata.freq,
     10     loss_type='l2',
     11     scaling=True,
     12     diff_steps=100,
     13     beta_end=0.1,
     14     beta_schedule="linear",
     15     trainer=Trainer(
     16         device=device,
     17         epochs=1,
     18         learning_rate=1e-3,
     19         num_batches_per_epoch=3,
     20         batch_size=64,
     21         )
     22 )
...
    343     object_setattr(__pydantic_self__, '__dict__', values)

ValidationError: 1 validation error for TimeGradEstimatorModel
scheduler
  field required (type=value_error.missing)

Any thoughts?

[kashif]

right you need also now give the esitmator the diffusers scheduler... see a running example here: https://github.com/kashif/time_match/blob/main/Time-Grad-Solar.ipynb

[kashif]

also don't think the trainer stuff from pts is needed as the 0.7.0 branch uses gluont's trainer

[simonyelisey]

@kashif above comments helped to move ahead, but I've got next error:

TypeError                                 Traceback (most recent call last)
Input In [35], in <cell line: 1>()
----> 1 predictor = estimator.train(electricity_dataset_train, cache_data=True, shuffle_buffer_length=1024)

File ~/anaconda3/envs/env1/lib/python3.9/site-packages/gluonts/torch/model/estimator.py:246, in PyTorchLightningEstimator.train(self, training_data, validation_data, shuffle_buffer_length, cache_data, ckpt_path, **kwargs)
    237 def train(
    238     self,
    239     training_data: Dataset,
   (...)
    244     **kwargs,
    245 ) -> PyTorchPredictor:
--> 246     return self.train_model(
    247         training_data,
    248         validation_data,
    249         shuffle_buffer_length=shuffle_buffer_length,
    250         cache_data=cache_data,
    251         ckpt_path=ckpt_path,
    252     ).predictor

File ~/anaconda3/envs/env1/lib/python3.9/site-packages/gluonts/torch/model/estimator.py:209, in PyTorchLightningEstimator.train_model(self, training_data, validation_data, from_predictor, shuffle_buffer_length, cache_data, ckpt_path, **kwargs)
    200 custom_callbacks = self.trainer_kwargs.pop("callbacks", [])
    201 trainer = pl.Trainer(
    202     **{
    203         "accelerator": "auto",
...
--> 132 raise TypeError(
    133     f"`model` must be a `LightningModule` or `torch._dynamo.OptimizedModule`, got `{type(model).__qualname__}`"
    134 )

TypeError: `model` must be a `LightningModule` or `torch._dynamo.OptimizedModule`, got `TimeGradLightningModule`

[kashif]

right cause pytorch-lighting has changed its API... so in the lightning module file of time-grad do:

from lightning import LightningModule

and then subclass from that:

class TimeGradLightningModule(LightningModule):

[simonyelisey]

https://github.com/zalandoresearch/pytorch-ts/issues/159#issuecomment-1908199600 helped to solve previous issue but now after all epochs I'm getting new error:

ValidationError                           Traceback (most recent call last)
Input In [39], in <cell line: 1>()
----> 1 predictor = estimator.train(electricity_dataset_train, cache_data=True, shuffle_buffer_length=1024)

File ~/anaconda3/envs/env1/lib/python3.9/site-packages/gluonts/torch/model/estimator.py:246, in PyTorchLightningEstimator.train(self, training_data, validation_data, shuffle_buffer_length, cache_data, ckpt_path, **kwargs)
    237 def train(
    238     self,
    239     training_data: Dataset,
   (...)
    244     **kwargs,
    245 ) -> PyTorchPredictor:
--> 246     return self.train_model(
    247         training_data,
    248         validation_data,
    249         shuffle_buffer_length=shuffle_buffer_length,
    250         cache_data=cache_data,
    251         ckpt_path=ckpt_path,
    252     ).predictor

File ~/anaconda3/envs/env1/lib/python3.9/site-packages/gluonts/torch/model/estimator.py:230, in PyTorchLightningEstimator.train_model(self, training_data, validation_data, from_predictor, shuffle_buffer_length, cache_data, ckpt_path, **kwargs)
    223 else:
    224     best_model = training_network
    226 return TrainOutput(
    227     transformation=transformation,
...
    343     object_setattr(__pydantic_self__, '__dict__', values)

ValidationError: 1 validation error for PyTorchPredictorModel
device
  str type expected (type=type_error.str)

Seems like I need to pass device argument but I can't understand where should I pass it to. Tried to pass it to estimator.train(device='cpu') but I got the same error.

[kashif]

hmm let me think i thought i fixed this issue as well... are you sure you are on the 0.7.0 branch?

[kashif]

actually try: pip install -U gluonts

[simonyelisey]

@kashif yes I'm on branch 0.7.0. https://github.com/zalandoresearch/pytorch-ts/issues/159#issuecomment-1910119265 didn't help.

[kashif]

so in the estimator set the device to a string:

        ...
        return PyTorchPredictor(
            input_transform=transformation + prediction_splitter,
            input_names=PREDICTION_INPUT_NAMES,
            prediction_net=module,
            batch_size=self.batch_size,
            prediction_length=self.prediction_length,
            device="cuda" if torch.cuda.is_available() else "CPU",
        )

[simonyelisey]

That works! Thank you @kashif so much!