Enhancing Prediction Performance and Monitoring for Large Datasets

[hnczj]

Hello,

I'm currently working with a dataset of over 500,000 rows and facing performance issues during the prediction phase with the Lag-Llama model. The process is considerably slow, and I'm exploring ways to enhance performance. Here are my specific concerns and questions:

Performance Optimization: Are there recommended strategies to speed up predictions for large datasets in Lag-Llama? Specifically, would techniques like batch processing or parallel computing be applicable, and how do they interact with the model's context length requirements? Are there any particular settings or configurations that could help improve performance without disrupting the time series continuity?

Progress Tracking: During the lengthy prediction process, it's crucial for me to monitor the progress effectively. I am looking for a method to track how many rows have been processed and how many predictions have been generated. I've considered tools like tqdm, but I'm unsure how well they integrate with Lag-Llama. Is there an internal feature within Lag-Llama that supports this functionality, or perhaps a recommended approach to implementing external tools without impacting performance?

I have previously attempted batch processing, but it seemed to disrupt the time series data and affect the label consistency due to the context length requirements.

Any advice or pointers to documentation would be greatly appreciated!

Thank you!

[ashok-arjun]

Hi,

1. For performance optimization, based on a recent update, you may try using use_single_pass_sampling to True when you create the estimator. This reduces the model to a deterministic point forecasting model, where only a single point is sampled from the distribution, and it is duplicated to create num_parallel_samples. Do note that the model was originally built for probabilistic forecasting, hence using it for point forecasting would be sub-optimal, and we have also not benchmarked the model for this task. However, it gives it a significant inference-time speedup. Depending on the accuracy - speedup tradeoff, you may or may not decide to use it :) But I'd suggest trying.

2. Progress tracking: There is no feature as of now that records the number of rows processed. We just pass a dataset, and the InstanceSampler as initialized here just samples windows from the dataset during training. To implement such a tool, you would have to override this InstanceSampler, and write your own where the windows processed are recorded or marked.

[hnczj]

您好，来信已收到，多谢！