search

ermongroup / CSDI

Codes for "CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation"
MIT License
261 stars 79 forks source link

Reproduction experiments #10

Closed HarperHao closed 5 months ago

HarperHao commented 6 months ago

Hi, Thanks your great work!

I ran the experiment on the physio dataset with the pre-training model file you gave, But I still can't get the results in your paper。

for example, at a 10% missing ratio, the results in the paper are: 0.498/0.217/0.238(RMSE/MAE/CRPS), but I get 0.552/0.25/0.28

Is this gap too big？

y-tashi commented 6 months ago

Hi, thanks for your interest!

I think the gap is too large. I have run the pretrained model today and got 0.234 CRPS. One of possible reasons is the number of samples. To reproduce the result, you should set "nsample = 100". Did you set nsample = 100? Another possible reason is the difference in environments such as the version of packages.

HarperHao commented 5 months ago

Thanks your help, I set "nsample=100", the results in the paper can be achieved!