search

gzerveas / mvts_transformer

Multivariate Time Series Transformer, public version
MIT License
718 stars 169 forks source link

Getting an error while running. Could anyone suggest where should I change the code #48

Closed Shubhamsg17 closed 1 year ago

Shubhamsg17 commented 1 year ago

Traceback (most recent call last): File "/content/drive/MyDrive/502/src/main.py", line 307, in main(config) File "/content/drive/MyDrive/502/src/main.py", line 235, in main aggr_metrics_val, best_metrics, best_value = validate(val_evaluator, tensorboard_writer, config, best_metrics, File "/content/drive/MyDrive/502/src/running.py", line 191, in validate aggr_metrics, per_batch = val_evaluator.evaluate(epoch, keep_all=True) File "/content/drive/MyDrive/502/src/running.py", line 451, in evaluate predictions = self.model(X.to(self.device), padding_masks) File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "/content/drive/MyDrive/502/src/models/ts_transformer.py", line 303, in forward output = self.transformer_encoder(inp, src_key_padding_mask=~padding_masks) # (seq_length, batch_size, d_model) File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/transformer.py", line 306, in forward output = mod(output, src_mask=mask, is_causal=is_causal, src_key_padding_mask=src_key_padding_mask_for_layers) File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, kwargs) TypeError: forward() got an unexpected keyword argument 'is_causal'

gzerveas commented 1 year ago

Looks like a package version problem. Can you install the packages from the failsafe requirements and try again?

Shubhamsg17 commented 1 year ago

let me try this solution and check. Thanks for you quick response

Shubhamsg17 commented 1 year ago

Tried with [failsafe requirements] getting new error:

2023-04-08 00:28:38,297 | INFO : Loading packages ... Traceback (most recent call last): File "/content/drive/MyDrive/502/src/main.py", line 29, in from running import setup, pipeline_factory, validate, check_progress, NEG_METRICS File "/content/drive/MyDrive/502/src/running.py", line 18, in import sklearn File "/usr/local/lib/python3.9/dist-packages/sklearn/init.py", line 80, in from .base import clone File "/usr/local/lib/python3.9/dist-packages/sklearn/base.py", line 21, in from .utils import _IS_32BIT File "/usr/local/lib/python3.9/dist-packages/sklearn/utils/init.py", line 22, in from .murmurhash import murmurhash3_32 File "sklearn/utils/murmurhash.pyx", line 1, in init sklearn.utils.murmurhash ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject

Shubhamsg17 commented 1 year ago

made changes in transformer.py file adding an extra parameter is_causal

DWCEN commented 1 year ago

Could you Please tell me how you resolve it？Thank you！

LGro commented 11 months ago

For the record in case other folks are running into this with newer pytorch versions, the required fix seems to be adding the is_causal: bool argument to the method TransformerBatchNormEncoderLayer.forward. This issue should only occur when running with --normalization_layer BatchNorm (default) and not with --normalization_layer LayerNorm.

aayushukla commented 4 months ago

Hi, set is_causal=None as argument to TransformerBatchNormEncoderLayer.forward. This made the code work for me.