search

DAMO-DI-ML / NeurIPS2023-One-Fits-All

The official code for "One Fits All: Power General Time Series Analysis by Pretrained LM (NeurIPS 2023 Spotlight)"
441 stars 61 forks source link

"test_data.inverse_transform" not found before test/evaluation for long-term forecasting? #43

Open wekaxx opened 2 months ago

wekaxx commented 2 months ago

Hi there,

When loading test datasets, your code used "sklearn.preprocessing.StandardScaler" to scale the original datasets. So, in order to produce the correct testing results, it is expected to scale the test-data back to the original values when calculating evaluation metrics? (the metrics should be evaluated based on the orginal values, not the scaled ones)

For example, I found "test_data.inverse_transform" was (correctly) done in TSLib (https://github.com/thuml/Time-Series-Library/blob/main/exp/exp_long_term_forecasting.py). However, I couldn't find similar treatments anywhere in your code?

Could you kindly point to where your code scaled the test-data back to original values before obtaining the final evaluation results? Much appreciated

tianzhou2011 commented 6 days ago

The reported MSE and MAE in the paper's table are relative values, so there’s no need to scale them back to calculate the testing values. However, if you wish to obtain the absolute MSE and MAE, you will need to perform that scaling.