Request to Add TimeMixer Model to neuralforecast Project

[kwuking]

Description

Request to Add TimeMixer Model to neuralforecast Project

Use case

Dear Maintainer,

I hope this message finds you well.

I am writing to you regarding your excellent work on the neuralforecast project, which provides scalable and user-friendly neural forecasting algorithms. Your contributions to the field are highly appreciated and have been incredibly valuable to the community.

We recently presented our work at ICLR 2024 on a new model called TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. This model leverages a decomposable multiscale mixing mechanism, achieving significant improvements in time series forecasting performance and demonstrating superior results across various benchmark tests.

Given the alignment of our work with the goals of the neuralforecast project, we would be honored if you could assist us in adding the TimeMixer model to the neuralforecast project. We believe that incorporating TimeMixer will further enhance the repository, providing the community with more options and stronger predictive capabilities.

The GitHub repository for TimeMixer can be found at the following link: https://github.com/kwuking/TimeMixer.

If you have any questions or require additional technical details, please do not hesitate to reach out. We are more than willing to provide any necessary support and assistance.

Thank you very much for your consideration and support. We look forward to your positive response.

[JQGoh]

@kwuking Thanks for sharing your work. I started to review the code implementation and learning the cited references. I am interested to contribute for this.

Once I am ready with the PR, I shall ask you and Nixtla's team to review.

[JQGoh]

@kwuking would love to learn about these

1. In TimeFeatureEmbedding, what is the rationale of using d_inp=1 for yearly (a), monthly (m) frequency time series? How did the team suggest the other d_inp settings like hourly using 4?
2. For TemporalEmbedding, can I assume that we only use FixedEmbedding for the TimeMixer model? Did not find that your team sets embed_type (so default behavior is fixed) parameter in the scripts used in https://github.com/thuml/Time-Series-Library

[kwuking]

@JQGoh Thank you for your support and assistance with our request. Here, we followed the approach of the Time-Series-Library project. Specifically:

1. In TimeFeatureEmbedding, the freq_map dictionary maps different time frequencies to an integer value, which represents the input dimension d_inp. For 'a' (representing year), d_inp is set to 1. This is because, in most time series analyses, the year is usually represented as a single value. For example, the year 2024 can be represented by a single integer.

Specifically, the value of d_inp determines the dimensionality of the input features. For example:

• 'h' (hour) is mapped to 4, possibly because hourly features might include hour, minute, second, and millisecond, covering multiple dimensions.
• 'd' (day) is mapped to 3, possibly because daily features might include year, month, and day, covering three dimensions.
• 'a' (year) is mapped to 1 because the year usually requires only a single value for representation.

Therefore, for yearly frequency, setting the input feature dimension d_inp to 1 is reasonable because the year typically only needs one value for representation, without requiring additional dimensions. This approach simplifies the model's input and reduces computational complexity.

2. For TemporalEmbedding, indeed, the embed_type is set to fixed here. Currently, we are using the same settings as the Time-Series-Library project.

Once again, we express our gratitude for your help and support. If you have any further questions, please feel free to contact me.

[kwuking]

@JQGoh Hi, are there any issues with TimeMixer at the moment? Thank you for your attention to TimeMixer. If you have any questions, please feel free to contact me at any time.

[JQGoh]

@kwuking Most recently I got new tasks assigned that I need to prioritize them. I expect to have more bandwidth during July.

I will keep you updated again when I am ready.

[kwuking]

@kwuking Most recently I got new tasks assigned that I need to prioritize them. I expect to have more bandwidth during July.

I will keep you updated again when I am ready.

Thank you very much for your help and support. We are looking forward to your progress. If there are any updates or if you have any questions, please don't hesitate to contact me. I noticed that you have also added our Time-LLM to neuralforecast. Once again, I would like to express my gratitude and regards to you.

[JQGoh]

@kwuking Thanks for this. I only added the support of new parameter for optimization related process. @marcopeix is the one who added Time-LLM to NeuralForecast.

[kwuking]

@kwuking Thanks for this. I only added the support of new parameter for optimization related process. @marcopeix is the one who added Time-LLM to NeuralForecast.

We greatly appreciate the tremendous support from you and @marcopeix for our work.. I have added acknowledgments to you on the Time-LLM GitHub homepage. The time series community greatly needs user-friendly framework and libraries, and neuralforecast has set a great example in this regard. This is crucial for promoting the prosperity and development of the community. We look forward to your further work and express our gratitude once again.