search

TuringLang / AdvancedHMC.jl

Robust, modular and efficient implementation of advanced Hamiltonian Monte Carlo algorithms
https://turinglang.org/AdvancedHMC.jl/
MIT License
228 stars 39 forks source link

Bayesian optimization based adaptation #254

Open Red-Portal opened 3 years ago

Red-Portal commented 3 years ago

Hi, Would you guys be interested in an implementation on the adaptative HMC scheme proposed in "Adaptive hamiltonian and riemann manifold monte carlo." ? I'm currently writing an implementation to go with AdvancedHMC.jl for my personal project, and if you guys are interested, I could contribute it.

AHMC involves fitting a Gaussian process, and running Bayesian optimization for maximizing the expected jumped squared distance. The former should require a GP or kernel learning library (my current implementation uses KernelFunctions.jl) and a simple global optimization algorithm like DIRECT (I'm using Nopt.jl).

Do you guys think it is worth adding considering the additional dependencies?

xukai92 commented 3 years ago

This looks like a great algorithm to have. You're welcome to fire a PR to add this.

Regarding the extra depedencies, what I would suggest is we host codes for this algorithm in the src/contrib folder and make the dependencies optional; see examples here. So there would be no real concern for extra dependencies.

I can offer more help once there is a PR :)