search

C-bowman / inference-tools

Python tools for Bayesian data analysis
MIT License
28 stars 3 forks source link

Can't use composite kernel with `GpOptimizer` #21

Open ZedThree opened 2 years ago

ZedThree commented 2 years ago

If I try to use a kernel like composite_kernel = SquaredExponential() + WhiteNoise() with GpOptimizer, the kernel evaluations have different shapes and I get the following traceback:

Traceback (most recent call last):
  File "/home/peter/Codes/gyrokinetics/gs2/work/s_alpha/./scan_with_gp.py", line 98, in <module>
    GPO.add_evaluation(new_x, new_y, new_y_err)
  File "/home/peter/.local/lib/python3.9/site-packages/inference/gp.py", line 829, in propose_evaluation
    proposed_ev, max_acq = self.multistart_bfgs()
  File "/home/peter/.local/lib/python3.9/site-packages/inference/gp.py", line 802, in multistart_bfgs
    results = [self.launch_bfgs(x0) for x0 in starting_positions]
  File "/home/peter/.local/lib/python3.9/site-packages/inference/gp.py", line 802, in <listcomp>
    results = [self.launch_bfgs(x0) for x0 in starting_positions]
  File "/home/peter/.local/lib/python3.9/site-packages/inference/gp.py", line 790, in launch_bfgs
    return fmin_l_bfgs_b(
  File "/usr/lib64/python3.9/site-packages/scipy/optimize/_lbfgsb_py.py", line 199, in fmin_l_bfgs_b
    res = _minimize_lbfgsb(fun, x0, args=args, jac=jac, bounds=bounds,
  File "/usr/lib64/python3.9/site-packages/scipy/optimize/_lbfgsb_py.py", line 308, in _minimize_lbfgsb
    sf = _prepare_scalar_function(fun, x0, jac=jac, args=args, epsilon=eps,
  File "/usr/lib64/python3.9/site-packages/scipy/optimize/_optimize.py", line 263, in _prepare_scalar_function
    sf = ScalarFunction(fun, x0, args, grad, hess,
  File "/usr/lib64/python3.9/site-packages/scipy/optimize/_differentiable_functions.py", line 158, in __init__
    self._update_fun()
  File "/usr/lib64/python3.9/site-packages/scipy/optimize/_differentiable_functions.py", line 251, in _update_fun
    self._update_fun_impl()
  File "/usr/lib64/python3.9/site-packages/scipy/optimize/_differentiable_functions.py", line 155, in update_fun
    self.f = fun_wrapped(self.x)
  File "/usr/lib64/python3.9/site-packages/scipy/optimize/_differentiable_functions.py", line 137, in fun_wrapped
    fx = fun(np.copy(x), *args)
  File "/usr/lib64/python3.9/site-packages/scipy/optimize/_optimize.py", line 76, in __call__
    self._compute_if_needed(x, *args)
  File "/usr/lib64/python3.9/site-packages/scipy/optimize/_optimize.py", line 70, in _compute_if_needed
    fg = self.fun(x, *args)
  File "/home/peter/.local/lib/python3.9/site-packages/inference/acquisition.py", line 98, in opt_func_gradient
    mu, sig = self.gp(x)
  File "/home/peter/.local/lib/python3.9/site-packages/inference/gp.py", line 180, in __call__
    K_qx = self.cov(q, self.x, self.cov_hyperpars)
  File "/home/peter/.local/lib/python3.9/site-packages/inference/covariance.py", line 67, in __call__
    return sum(
ValueError: operands could not be broadcast together with shapes (1,2) (2,4)
C-bowman commented 2 years ago

@ZedThree This one is a little confusing - I think it's a problem with how the GpRegressor object that's held inside the instance of GpOptimizer is parsing the input data.

For example, if I take this demo script in the repo, and I swap the kernel to SquaredExponential() + WhiteNoise(), it fails in the way I'd expect (because CompositeCovariance is missing some methods required by GpOptimizer):

Traceback (most recent call last):
  File "/home/cbowman/inference-tools/demos/scripts/GpOptimiser_demo.py", line 116, in <module>
    new_x = GP.propose_evaluation()
  File "/home/cbowman/inference-tools/inference/gp.py", line 829, in propose_evaluation
    proposed_ev, max_acq = self.multistart_bfgs()
  File "/home/cbowman/inference-tools/inference/gp.py", line 802, in multistart_bfgs
    results = [self.launch_bfgs(x0) for x0 in starting_positions]
  File "/home/cbowman/inference-tools/inference/gp.py", line 802, in <listcomp>
    results = [self.launch_bfgs(x0) for x0 in starting_positions]
  File "/home/cbowman/inference-tools/inference/gp.py", line 790, in launch_bfgs
    return fmin_l_bfgs_b(
  File "/home/cbowman/.local/lib/python3.9/site-packages/scipy/optimize/lbfgsb.py", line 197, in fmin_l_bfgs_b
    res = _minimize_lbfgsb(fun, x0, args=args, jac=jac, bounds=bounds,
  File "/home/cbowman/.local/lib/python3.9/site-packages/scipy/optimize/lbfgsb.py", line 306, in _minimize_lbfgsb
    sf = _prepare_scalar_function(fun, x0, jac=jac, args=args, epsilon=eps,
  File "/home/cbowman/.local/lib/python3.9/site-packages/scipy/optimize/optimize.py", line 261, in _prepare_scalar_function
    sf = ScalarFunction(fun, x0, args, grad, hess,
  File "/home/cbowman/.local/lib/python3.9/site-packages/scipy/optimize/_differentiable_functions.py", line 140, in __init__
    self._update_fun()
  File "/home/cbowman/.local/lib/python3.9/site-packages/scipy/optimize/_differentiable_functions.py", line 233, in _update_fun
    self._update_fun_impl()
  File "/home/cbowman/.local/lib/python3.9/site-packages/scipy/optimize/_differentiable_functions.py", line 137, in update_fun
    self.f = fun_wrapped(self.x)
  File "/home/cbowman/.local/lib/python3.9/site-packages/scipy/optimize/_differentiable_functions.py", line 134, in fun_wrapped
    return fun(np.copy(x), *args)
  File "/home/cbowman/.local/lib/python3.9/site-packages/scipy/optimize/optimize.py", line 74, in __call__
    self._compute_if_needed(x, *args)
  File "/home/cbowman/.local/lib/python3.9/site-packages/scipy/optimize/optimize.py", line 68, in _compute_if_needed
    fg = self.fun(x, *args)
  File "/home/cbowman/inference-tools/inference/acquisition.py", line 99, in opt_func_gradient
    dmu, dvar = self.gp.spatial_derivatives(x)
  File "/home/cbowman/inference-tools/inference/gp.py", line 380, in spatial_derivatives
    A, _ = self.cov.gradient_terms(q[0, :], self.x, self.cov_hyperpars)
  File "/home/cbowman/inference-tools/inference/covariance.py", line 32, in gradient_terms
    raise NotImplementedError(
NotImplementedError: 
            Gradient calculations are not yet available for the
            <class 'inference.covariance.CompositeCovariance'> covariance function.

Process finished with exit code 1

This means once the bug is fixed, it'll still fail but at least with a proper error message! I can implement the missing code to get GpOptimizer to work with a composite covariance function if it's useful to you though - I've been meaning to do it for a while.

I can try to reproduce the bug by playing around with the input data shape / type, but if you could send me a script which crashes with the bug that would be easiest.

ZedThree commented 2 years ago

I'll see if I can reduce my script down a bit -- the objective function is a Fortran code, so it's not something straight forward to share.

This was just something I was playing about with, so it's not terribly urgent.