search

TuringLang / AdvancedHMC.jl

Robust, modular and efficient implementation of advanced Hamiltonian Monte Carlo algorithms
https://turinglang.org/AdvancedHMC.jl/
MIT License
228 stars 39 forks source link

Initial Epsilon in find_good_step_size function #216

Open RajDandekar opened 3 years ago

RajDandekar commented 3 years ago

I am trying to integrate AdvancedHMC.jl in trying to find terms of a differential equation (in the spirit of Universal Ordinary Differential Equations (https://arxiv.org/abs/2001.04385)

I was curious to learn more about the initialization: ϵ′ = ϵ = T(0.1) in the find_good_step_size function. I noticed that using such a large value caused the derivative of the hamiltonian (dH/ dtheta) to blow up leading to a lot of numerical errors before finding the initial step size.

I found that when I initialize it to a smaller value (ϵ′ = ϵ = T(0.001)), I get about a 50 times performance improvement in the find_good_step_size function.

Any comments on this?

I can share my code if that would help.

Thanks!

Vaibhavdixit02 commented 3 years ago

I find this peculiar also because trying it out with Neural ODEs (and DEs in the tutorial) the step sizes in the range of 0.25 gave much more well behaved sampling. It might be worthwhile to expose this since it varies between problems?

xukai92 commented 3 years ago

Feel free to fire a PR to allow optional initial step size in find_good_step_size.

Vaibhavdixit02 commented 3 years ago

I know @KumarRishabh was looking for a first issue, this should be easy enough if he wants to take a stab

KumarRishabh commented 3 years ago

Thanks @Vaibhavdixit02! Looks like a good issue to get started with. Will start working on this.