search

KimMeen / Time-LLM

[ICLR 2024] Official implementation of " 🦙 Time-LLM: Time Series Forecasting by Reprogramming Large Language Models"
https://arxiv.org/abs/2310.01728
Apache License 2.0
1.01k stars 177 forks source link

Mamba or Jamba models #61

Open DewEfresh opened 2 months ago

DewEfresh commented 2 months ago

Has any work been done with state space models. I'd be curious how they would perform with this framework applied.

kwuking commented 2 months ago

Clearly, this is a very interesting question. The use of state-space models as a base for large models has recently become a very popular topic. For example, papers like "Is Mamba Effective for Time Series Forecasting?" and "TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting" have made numerous attempts. In fact, we have also been studying the use of state-space models for time series forecasting for a long time and have followed the precursor to the Mamba model, the S4 model. However, we have not achieved good results in time series forecasting, which is why we decided to "let the bullets fly" for a while. We also look forward to seeing breakthroughs in time series forecasting using SSM. Finally, I would like to mention that as a general reprogramming framework, we are obviously also capable of adapting Mamba or Jamba and look forward to more like-minded friends joining us to contribute code.