search

Nixtla / neuralforecast

Scalable and user friendly neural :brain: forecasting algorithms.
https://nixtlaverse.nixtla.io/neuralforecast
Apache License 2.0
2.93k stars 337 forks source link

[Models] Evaluate `torch.compile` on `nf` models #626

Closed cchallu closed 3 weeks ago

cchallu commented 1 year ago

Description

The torch.compile method has been introduced in PyTorch 2.0 and is aimed at optimizing and accelerating the execution of PyTorch models. The NeuralForecast library, which leverages PyTorch for time series forecasting, has the potential to benefit from this new feature. However, it is essential to thoroughly test the torch.compile method to ensure its compatibility, effectiveness, and performance gains.

We kindly request users and contributors to test the torch.compile method with NeuralForecast models and provide feedback on their experiences. The feedback will help us assess the viability and potential improvements of utilizing this method in the library.

Testing Guidelines:

  1. Fork or clone the repository.
  2. Create a new branch to work on your tests: git checkout -b torch-compile-testing
  3. Implement the necessary changes to integrate the torch.compile method into nf models (see https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.html).
  4. Choose time series datasets from datasetsforecast package (tutorials on how to download are available in our documentation) or use other publicly available datasets.
  5. Train nf models using the torch.compile method and evaluate their performance against the original models without compilation.
  6. If you encounter any issues, bugs, or unexpected behavior related to the torch.compile method, please create a new issue, providing detailed steps to reproduce the problem.

Feedback and Reporting: We encourage you to share your findings, observations, and any potential issues or improvements discovered during the testing process in this issue or in our slack channels.

Acknowledgment: We highly appreciate your time and effort in testing the torch.compile method with nf models. Your feedback will contribute to the improvement of the library and the PyTorch ecosystem as a whole. Thank you for your valuable support!

Please feel free to reach out if you have any questions or need further assistance.

AzulGarza commented 7 months ago

@quest-bot stash 400

quest-bot[bot] commented 7 months ago

New Quest! image New Quest!

A new Quest has been launched in @Nixtla’s repo. Merge a PR that solves this issue to loot the Quest and earn your reward.

Loot of 400 USD has been stashed in this issue to reward the solver!

🗡 Comment @quest-bot embark to check-in for this Quest and start solving the issue. Other solvers will be notified!

⚔️ When you submit a PR, comment @quest-bot loot #626 to link your PR to this Quest.

Questions? Check out the docs.

patel-zeel commented 7 months ago

@quest-bot embark

Hi @cchallu,

I am excited to get started with my first issue on neuralforecast. Some quick questions before taking a deep dive:

quest-bot[bot] commented 7 months ago

@patel-zeel has embarked on their Quest 🗡

Questions? Check out the docs.

cchallu commented 5 months ago

Hi @patel-zeel. Sorry for the delay on the answer. Yes, we would like to understand if it also improves on GPU. Can you try using Colab? We can add the cost of the GPU to the reward.

patel-zeel commented 5 months ago

Hi @cchallu, thank you for confirmation. I am planning to evaluate this on Nvidia Quadro RTX 5000 (16 GB). To bring in an essential detail in conversation, the following note is mentioned in torch.compile tutorial.

NOTE: a modern NVIDIA GPU (H100, A100, or V100) is recommended for this tutorial in order to reproduce the speedup numbers shown below and documented elsewhere.

Is neuralforecast's goal to leverage torch.complie for most of the GPU cards or only high-end GPUs like H/A/V100? What'd be the next step if the speedup is not possible with common GPUs (other than H/A/V100)?

quest-bot[bot] commented 3 months ago

Swords down 🗡

❌ The Quest on issue #626 has been aborted by @Nixtla

Head over to https://quine.sh/quests/solver to find other active Quests.

cc @patel-zeel

This Quest has been closed ⚔️.

Questions? Check out the docs