lss-1138 / SparseTSF

[ICML 2024 Oral] Official repository of the SparseTSF paper: "SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters". This work is developed by the Lab of Professor Weiwei Lin (linww@scut.edu.cn), South China University of Technology; Pengcheng Laboratory.
Apache License 2.0
122 stars 7 forks source link

Is the paper in arXiv final version? #5

Closed mb-Ma closed 4 months ago

mb-Ma commented 4 months ago

I noticed that SparseTSF adopts the channel-independent strategy in Section 3.1.

However, The channel-dependent strategy is used in the source code.

Is there a conflict between them?

lss-1138 commented 4 months ago

Thank you for your question. The channel-independent strategy used in the source code models/SparseTSF.py is indeed a channel-independent approach.

In fact, there are two types of channel-independent strategies: one involves modeling with shared parameters between channels (as used in PatchTST), and the other involves modeling with non-shared parameters between channels (completely channel-independent), which means using independent models for each channel (e.g., setting the parameter individual to True in the DLinear model).

In SparseTSF, we use the former channel-independent strategy, where parameters are shared between each channel. This is consistent with the description in Section 3.1, which states: "the CI method finds a shared function $f$ for each univariate series." There is no modeling of relationships between channels involved here.

mb-Ma commented 4 months ago

Thank you for your quick feedback. But, confusedly, it seems a conflict if shared and non-shared parameters both are channel-independent strategies. In other words, what is the channel-dependent strategy in the context?

Could you explain more, please?

lss-1138 commented 4 months ago

A channel-dependent strategy, such as the ones adopted by Informer and Autoformer, mixes the data from different channels at the model's input. Additionally, models like Crossformer and iTransformer explicitly allow data interaction between channels during modeling. Therefore, the key distinction between channel-independent and channel-dependent strategies is whether there is interaction between the information of different channels. The channel-independent strategy with shared parameters does not involve interaction between channels.

I recommend reading this paper: The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting. This paper provides a detailed study on the differences, advantages, and disadvantages of channel-dependent and channel-independent strategies.

mb-Ma commented 4 months ago

I agree with the claim that distinguishes channel-dependent and -independent strategies from the interaction perspective. Personality, the share-parameter strategy is an implicit interaction between channels.

I will check this from the paper that you mentioned above. Thank you for your reply, again. :-)