search

SheffieldML / GPy

Gaussian processes framework in python
BSD 3-Clause "New" or "Revised" License
2.01k stars 557 forks source link

What hyperparameter optimisation method is used by GPy #680

Open kentcov opened 5 years ago

kentcov commented 5 years ago

I couldn't find anything about the technique used by GPy when optimising the length-scales in an RBF kernel with ARD=True.

I'm aware of Log-likelihood optimisation (methods such as conjugate gradient) or Integration via Hybrid Monte Carlo methods

(Both detailed in Williams & Rasmussen - Gaussian Processes for Regression 1996)

is one of these used, or is there another approach being used here?

kind regards,

xorb0ss commented 5 years ago

I believe it's Limited-memory BFGS (L-BFGS) or a variation.

lawrennd commented 5 years ago

By default it's maximum likelihood by L-BFGS, but you can change the optimizer

and there are details on HMC in this notebook:

http://nbviewer.jupyter.org/github/SheffieldML/notebook/blob/master/GPy/sampling_hmc.ipynb

On Tue, Sep 25, 2018 at 7:34 PM xorb0ss notifications@github.com wrote:

I believe it's Limited-memory BFGS (L-BFGS) or a variation.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SheffieldML/GPy/issues/680#issuecomment-424453828, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIKWg-OXs9izok24r7Q5f1IjtZv9s1Xks5uendOgaJpZM4W06FM .