Informer achieves lower MSEs in their own paper

[limjcst]

Taking the Electricity dataset (named ECL in the Informer paper) as an example, Informer reported an MSE of 0.540 for 720-step prediction, while the Pyraformer reported an MSE of 4.365 for Informer. What's more, another paper (Autoformer published in NIPS'21) reported an MSE of 0.373 for Informer, which is even lower than the Informer paper. The same thing happens for LogTrans.

I am wondering,

• Does Pyraformer calculate MSE/MAE in a different way from Informer?
• If so, could you please share your considerations for the choice?

[Zhazhan]

Informer and Pyraformer preprocess the electricity dataset in different ways. We followed DeepAR in preprocessing the electricity dataset (the scripts are provided in this repo), while Informer appears to preprocess it in another way. As a result, the datasets we use are quite different. We re-ran Informer on our preprocessed data and reported the results.

[limjcst]

Thanks for your quick response!