search

rtqichen / torchdiffeq

Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation.
MIT License
5.57k stars 925 forks source link

RMSE issue in latent_ode #117

Open esther-soyoung opened 4 years ago

esther-soyoung commented 4 years ago

Hello!
I have a question whether the released version of latent_ode.py
is identical with the one used for experiments on the paper.

I was playing around with latent_ode.py and confronted an issue reproducing RMSE on the paper.
Based on the paper, predictive RMSE on test set should be around 0.13~0.16,

Screen Shot 2020-08-19 at 2 01 50 PM

but I only get RMSE of 0.3~0.4; around 0.3 for validation(interpolation) set, around 0.4 for test(extrapolation) set.
Didn't modify any hyperparameter, running on torch==1.5.1, torchdiffeq==0.0.1.

Could you please check this out?
Please let me know if there's any extra work needed to get this fixed.
Thanks in advance:)

esther-soyoung commented 4 years ago

Think I found a way around; adjusting(or removing) gaussian noise yields RMSE score similar to the one from the paper.

klauscc commented 3 years ago

Hi, @esther-soyoung, I am also playing with latent_ode.py. But the extrapolation is very unstable (sometimes good sometings very bad). Do you mind sharing your modification to the original code? Moreover, in the paper Tab. 2 is for irregular sampling, but in the code the samples are evenly sampled with different start points.