search

google-research / torchsde

Differentiable SDE solvers with GPU support and efficient sensitivity analysis.
Apache License 2.0
1.51k stars 194 forks source link

Query on KL_Scheduler #118

Closed PANG-98 closed 1 year ago

PANG-98 commented 1 year ago

Hey fellows, excellent work and really appreciate the released code. But, I'm a little bit confused about the "kl_scheduler" used in "latent_sde_lorenz.py". What does it mean, and do I really need it ?

lxuechen commented 1 year ago

Part of this is due to legacy code that we wanted to support still. It's not strictly necessary from a functionality perspective.

The point of this was that you could have a setting where you'd like to weight the penalty incurred by different parts of the KL objective differently.

Say you solve from t0 to t1 and then to t2. You'd get 2 KL penalties -- one for the part between t0 to t1, and another from t1 to t2. kl_scheduler allows you then to adjust the weight assigned to the two components.