Fig. 1: An overview of TSDiff’s use cases. Predict: By utilizing observation self-guidance, TSDiff can be
conditioned during inference to perform predictive tasks such as forecasting. Refine: Predictions
of base forecasters can be improved by leveraging the implicit probability density of TSDiff.
Synthesize: Realistic samples generated by TSDiff can be used to train downstream forecasters achieving good
performance on real test data.
This repository contains the official implementation of the NeurIPS 2023 paper Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting. In this paper, we propose TSDiff, an unconditional diffusion model for time series. Our proposed self-guidance mechanism enables conditioning TSDiff for downstream tasks during inference, without requiring auxiliary networks or altering the training procedure. Furthermore, our refinement scheme leverages the implicit density learned by the diffusion model to iteratively refine the predictions of base forecasters. Finally, we demonstrate the high quality of the synthetic time series by training downstrain models solely on generated data and introducing the Linear Predictive Score (LPS).
Fig. 2: Example forecasts generated by TSDiff-Q for
time series in Electricity, KDDCup, and Exchange — three datasets with different frequencies and/or prediction lengths.
TSDiff requires Python 3.8 or higher.
conda create --name tsdiff --yes python=3.8 && conda activate tsdiff
pip install --editable "."
[!TIP]
We have some updates in theupdate
branch. If you're interested in testing out TSDiff or training it on a custom dataset, using theupdate
branch maybe faster for training.
Train models using the train_model.py
and train_cond_model.py
scripts for TSDiff
and TSDiff-Cond
, respectively. Sample configurations can be found in configs/train_tsdiff.yaml
and configs/train_tsdiff-cond.yaml
. Specific configurations used in the paper can be found in configs/train_tsdiff
and configs/train_tsdiff-cond
.
Example commands for regular (i.e., no missing values) forecasting:
# Train TSDiff on the Uber dataset for regular forecasting
python bin/train_model.py -c configs/train_tsdiff/train_uber_tlc.yaml
# Train TSDiff on the M4 dataset for regular forecasting
python bin/train_model.py -c configs/train_tsdiff/train_m4.yaml
# Train TSDiff-Cond on the Uber dataset for regular forecasting
python bin/train_cond_model.py -c configs/train_tsdiff-cond/uber_tlc_hourly.yaml
# Train TSDiff-Cond on the M4 dataset for regular forecasting
python bin/train_cond_model.py -c configs/train_tsdiff-cond/m4_hourly.yaml
Example commands for forecasting with missing values:
# Train TSDiff on the Uber dataset for the missing values experiment
python bin/train_model.py -c configs/train_tsdiff/train_missing_uber_tlc.yaml
# Train TSDiff on the KDDCup dataset for the missing values experiment
python bin/train_model.py -c configs/train_tsdiff/train_missing_kdd_cup.yaml
# Train TSDiff-Cond on the Uber dataset for the RM missing values experiment
python bin/train_cond_model.py -c configs/train_tsdiff-cond/missing_RM_uber_tlc_hourly.yaml
# Train TSDiff-Cond on the KDDCup dataset for the BM-B missing values experiment
python bin/train_cond_model.py -c configs/train_tsdiff-cond/missing_BM-B_kdd_cup_2018_without_missing.yaml
# Train TSDiff-Cond on the KDDCup dataset for the BM-E missing values experiment
python bin/train_cond_model.py -c configs/train_tsdiff-cond/missing_BM-E_kdd_cup_2018_without_missing.yaml
Note that for TSDiff we train only one model and all the missing value scenarios are evaluated using the same unconditional model. However, for TSDiff-Cond, one model is trained per missingness scenario.
The unconditional models trained above can be used for the following tasks.
Use the guidance_experiment.py
script and configs/guidance.yaml
config to run the forecasting experiments. Specific configurations used in the paper can be found in configs/guidance/
.
Example commands:
# Run observation self-guidance on the Solar dataset
python bin/guidance_experiment.py -c configs/guidance/guidance_solar.yaml --ckpt /path/to/ckpt
# Run observation self-guidance on the KDDCup dataset
python bin/guidance_experiment.py -c configs/guidance/guidance_kdd_cup.yaml --ckpt /path/to/ckpt
Use refinement_experiment.py
script and configs/refinement.yaml
config to run the refinement experiments. Specific configurations used in the paper can be found in configs/refinement/
.
Example commands:
# Refine predictions from the Linear model on the Solar dataset
python bin/refinement_experiment.py -c configs/refinement/solar_nips-linear.yaml --ckpt /path/to/ckpt
# Refine predictions from the DeepAR model on the M4 dataset
python bin/refinement_experiment.py -c configs/refinement/m4_hourly-deepar.yaml --ckpt /path/to/ckpt
Use tstr_experiment.py
script and configs/tstr.yaml
config to run the train on synthetic-test on real experiments. Specific configurations used in the paper can be found in configs/tstr/
.
Example commands:
# TSTR on the Solar Dataset
python bin/tstr_experiment.py -c configs/tstr/solar_nips.yaml --ckpt /path/to/ckpt
# TSTR on the KDDCup Dataset
python bin/tstr_experiment.py -c configs/tstr/kdd_cup_2018_without_missing.yaml --ckpt /path/to/ckpt
If you find this repository or the ideas presented in our paper useful, please consider citing.
@inproceedings{kollovieh2023predict,
author = {Kollovieh, Marcel and Ansari, Abdul Fatir and Bohlke-Schneider, Michael and Zschiegner, Jasper and Wang, Hao and Wang, Yuyang},
title = {Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting},
booktitle = {Advances in Neural Information Processing Systems},
year = {2023}
}
See CONTRIBUTING for more information.
This project is licensed under the Apache-2.0 License.