search

cornellius-gp / gpytorch

A highly efficient implementation of Gaussian Processes in PyTorch
MIT License
3.52k stars 552 forks source link

SCG method in GPyTorch #1627

Open takafusui opened 3 years ago

takafusui commented 3 years ago

Hi,

Are there any implementations where the scaled conjugate gradient (: SCG) method is used to optimize the model's hyperparameters, instead of LBFGS or the stochastic gradient descent?

jacobrgardner commented 3 years ago

No, primarily because there weren't any good implementations of nonlinear conjugate gradients last time we looked.

Is there a reason you need this specifically over something like LBFGS?

takafusui commented 3 years ago

Thank you @jacobrgardner for your answer. Not really. I usually use either adam or PyTorch-LBFGS to train GP. I want to try SCG and compare it with the other two optimizers.

gpleiss commented 3 years ago

@takafusui we're open to a PR, if you'd like to implement it! 😉