Optimizing inducing points in EP_FITC?

[jameshensman]

Hi pyGPs devs!

I'm trying to build a classifier with the EP_FITC method. I've read the docs, which are pretty clear, but I don't seem to be able to optimise the inducing variables. Here's my code:

import numpy as np import pyGPs Xtrain = np.random.randn(100,1) Ytrain = np.random.randint(0,2,(100,1)) Z = np.random.randn(10,1) m_GFITC = pyGPs.GPC_FITC() m_GFITC.setPrior(mean=pyGPs.mean.Zero(), kernel=pyGPs.cov.RBF(), inducing_points=Z) m_GFITC.setData(Xtrain, np.where(Ytrain==1, 1, -1)) m_GFITC.optimize()

I think this replicates the doc at http://www-ai.cs.uni-dortmund.de/weblab/static/api_docs/pyGPs/GPC_FITC.html but I get the error

in () ----> 1 m_GFITC.optimize() /Users/james/work/varGP/pyGPs/pyGPs/Core/gp.pyc in optimize(self, x, y) 187 188 # optimize --> 189 optimalHyp, optimalNlZ = self.optimizer.findMin(self.x, self.y) 190 self.nlZ = optimalNlZ 191 /Users/james/work/varGP/pyGPs/pyGPs/Core/opt.pyc in findMin(self, x, y) 229 self.errorCounter += 1 230 if not self.searchConfig: --> 231 raise Exception("Can not use minimize. Try other hyparameters") 232 self.trailsCounter += 1 233 Exception: Can not use minimize. Try other hyparameters Any clues? I'd really like to be able to optimise the inducing poins in FITC. Best wishes, James.

[mathDR]

Hey James, I am looking at this right now. Thanks for pointing it out.

[shansfolder]

Hi James,

First of all, inducing points will not be optimized in our package anyway. Once you set the inducing points, it is fixed during the optimization. The optimization process will only optimize covariance/mean/likelihood hyper parameters.

To my knowledge, this error usually occurs when the local optimization method does not fit with your given dataset and/or gp settings. You can either try with another optimization method for example: setOptimizer("SCG")

More optimization methods see http://www-ai.cs.uni-dortmund.de/weblab/static/api_docs/pyGPs/Opts.html

Another way is to use another covariance function. i.e. try other hyper parameters. And yes I agree that the error message we have is quite confusing. We should tell users in a clear way.

Thank you for pointing it out. I hope this will solve your problem. If not, feel free to ask again.

Cheers, Shan

On 2014-10-11, at 下午9:29, James Hensman notifications@github.com wrote:

Hi pyGPs devs!

I'm trying to build a classifier with the EP_FITC method. I've read the docs, which are pretty clear, but I don't seem to be able to optimise the inducing variables. Here's my code:

import numpy as np import pyGPs Xtrain = np.random.randn(100,1) Ytrain = np.random.randint(0,2,(100,1)) Z = np.random.randn(10,1) m_GFITC = pyGPs.GPC_FITC() m_GFITC.setPrior(mean=pyGPs.mean.Zero(), kernel=pyGPs.cov.RBF(), inducing_points=Z) m_GFITC.setData(Xtrain, np.where(Ytrain==1, 1, -1)) m_GFITC.optimize()

I think this replicates the doc at http://www-ai.cs.uni-dortmund.de/weblab/static/api_docs/pyGPs/GPC_FITC.html but I get the error

in () ----> 1 m_GFITC.optimize()

/Users/james/work/varGP/pyGPs/pyGPs/Core/gp.pyc in optimize(self, x, y) 187 188 # optimize --> 189 optimalHyp, optimalNlZ = self.optimizer.findMin(self.x, self.y) 190 self.nlZ = optimalNlZ 191

/Users/james/work/varGP/pyGPs/pyGPs/Core/opt.pyc in findMin(self, x, y) 229 self.errorCounter += 1 230 if not self.searchConfig: --> 231 raise Exception("Can not use minimize. Try other hyparameters") 232 self.trailsCounter += 1 233

Exception: Can not use minimize. Try other hyparameters

Any clues? I'd really like to be able to optimise the inducing poins in FITC.

Best wishes, James.

— Reply to this email directly or view it on GitHub.

[mathDR]

James, optimizing the inducing input locations seems like a great feature to have. I will look at the various gradient functions required and start the process of adding this feature this week.

Dan