Added hyperparameter optimization example to README

[wesselb]

Thanks for this example! This looks great. How would you feel about the following simplification of the code, which removes some unused components?

import lab.tensorflow as B
import tensorflow as tf
import wbml.out as out
from varz.spec import parametrised, Positive
from varz.tensorflow import Vars, minimise_l_bfgs_b

from stheno import EQ, GP

# Define some points to sample data at.
x_obs = B.linspace(0, 2, 50)

# Sample a true, underlying function with length scale `0.2` and observations with noise
# variance `0.05`. We will try to learn these parameters from the observations.
f = GP(EQ().stretch(0.2))
y_obs = f(x_obs, 0.05).sample()

@parametrised
def model(vs, noise: Positive = 0.1, length_scale: Positive = 1):
    """Constuct a model with learnable parameters."""
    return GP(EQ().stretch(length_scale)), noise

def objective(vs):
    """The objective function to optimise is the negative log-marginal likelihood."""
    f, noise = model(vs)
    return -f(x_obs, noise).logpdf(y_obs)

# Perform optimisation.
vs = Vars(tf.float64)
minimise_l_bfgs_b(objective, vs, trace=True)

# Print the learned parameters.
with out.Section("Learned parameters"):
    vs.print()

Minimisation of "objective":
    Iteration 1/1000:
        Time elapsed: 0.0 s
        Time left:  24.2 s
        Objective value: 55.06
    Iteration 9/1000:
        Time elapsed: 0.1 s
        Time left:  7.9 s
        Objective value: 24.9
    Done!
Termination message:
    CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
Learned parameters:
    noise:      0.06494
    length_scale: 0.206

[patel-zeel]

Yeah, this looks better. However can you please explain what exact role @parametrised plays here and how it gets related with vs?

[wesselb]

@parametrised
def model(vs, noise: Positive = 0.1, length_scale: Positive = 1):
    """Constuct a model with learnable parameters."""
    return GP(EQ().stretch(length_scale)), noise

is equivalent to

def model(vs):
    """Constuct a model with learnable parameters."""
    noise = vs.positive(0.1, name="noise")
    length_scale = vs.positive(1, name="length_scale")
    return GP(EQ().stretch(length_scale)), noise

This conversion is performed by @parametrised. I'm happy to go with either version! I agree that using @parametrised may be a little bit too much syntactic sugar.

[patel-zeel]

Yes, I agree. Given that the examples in README help newcomers the most, explicit is better than implicit. @parametrised function can be kept as a comment just below the simple function.

[wesselb]

Perfect. Let's go with the more explicit version then. (I think CI is failing because it's currently behind master, which incorporates some missing dependencies.)

[patel-zeel]

I lost this fork so reopening this PR at #18.