search

cure-lab / SCINet

The GitHub repository for the paper: “Time Series is a Special Sequence: Forecasting with Sample Convolution and Interaction“. (NeurIPS 2022)
Apache License 2.0
620 stars 127 forks source link

Unable to reproduce the results #2

Closed ZhangTP1996 closed 2 years ago

ZhangTP1996 commented 2 years ago

I am surprised by the excellent performance of SCINet reported in the paper. However, I cannot reproduce the results. I download the code and use the same command and try to reproduce the results of the exchange dataset when the lookahead is 3. The result is rse 0.018 and corr 0.9738, which is greatly different from the result reported in the paper (rse 0.0147 and corr 0.9868). Could you please tell me what happens?

AlexMinhao commented 2 years ago

We do apologize for bringing you this trouble. We have found a bug in the Solar, Traffic, Electricity and Exchange-rate data loader, which will affect the results. Now, the bug has been fixed, but the hyper-parameters need tuning. We will update the results and paper as soon as possible. Thanks a lot.

ZhangTP1996 commented 2 years ago

We do apologize for bringing you this trouble. We have found a bug in the Solar, Traffic, Electricity and Exchange-rate data loader, which will affect the results. Now, the bug has been fixed, but the hyper-parameters need tuning. We will update the results and paper as soon as possible. Thanks a lot.

Is the bug fixed in the current version of the codes in github?

AlexMinhao commented 2 years ago

We do apologize for bringing you this trouble. We have found a bug in the Solar, Traffic, Electricity and Exchange-rate data loader, which will affect the results. Now, the bug has been fixed, but the hyper-parameters need tuning. We will update the results and paper as soon as possible. Thanks a lot.

Is the bug fixed in the current version of the codes in github?

Yes, the bug has already been fixed in the current version, and we will update the training commands as soon as possible. Thanks.