search

yuqinie98 / PatchTST

An offical implementation of PatchTST: "A Time Series is Worth 64 Words: Long-term Forecasting with Transformers." (ICLR 2023) https://arxiv.org/abs/2211.14730
Apache License 2.0
1.37k stars 248 forks source link

Input normalization twice - scaler and revin #33

Closed ikvision closed 1 year ago

ikvision commented 1 year ago

While loading the data there is zcore normalization z = (x - mean) / std of the input data: https://github.com/yuqinie98/PatchTST/blob/de8d7f0da12f4af1bfe13de3d7fe0b888bd84ea9/PatchTST_self_supervised/src/data/pred_dataset.py#L48 https://github.com/yuqinie98/PatchTST/blob/de8d7f0da12f4af1bfe13de3d7fe0b888bd84ea9/PatchTST_self_supervised/src/data/pred_dataset.py#L65 Before the forward path there is a zscore normalization to input as part of revin layer: https://github.com/yuqinie98/PatchTST/blob/de8d7f0da12f4af1bfe13de3d7fe0b888bd84ea9/PatchTST_self_supervised/src/models/layers/revin.py#L37-L39

Questions:

  1. Can you help me understand the difference between the two normalizations and why both are required by default?
  2. Why in finetune RevInCB gets denorm=True while in pretrain RevInCB gets denorm=False?
yuqinie98 commented 1 year ago

Hello, actually if you have instance normalization (revin), you can ignore standard normalization method. The difference is that the former normalize on each instance, the latter one normalize on the whole training dataset. More details are explained in this paper: https://github.com/ts-kim/RevIN. For evaluation, the data is normalized, so no denorm in output for finetune.