Quadratic mean for GP models

Overview

Standard version

$$ \mu (x) = x A x^\top, $$

$$ \nabla \mu (x) = \frac12 x (A + A^\top), $$

$$ \nabla^2 \mu(x) = A. $$

Positive version

We define $A = L L^\top$ and $L$ is a lower triangular matrix. We also that the gradient becomes $\nabla \mu(x) = x A$ as A is now symmetric, and the Hessian remains unchanged.

Linear operations

Add the possibilities to do linear operations with means. For all mean modules $\mu, \nu$ and for all $\lambda \in \mathbb{R}$, we can define the following mean:

$$ \eta(x) := \mu(x) + \lambda \cdot \nu(x) $$

$$ = \mu(x) + \nu(x) \cdot \lambda. $$

It is especially useful to combine a linear mean with a quadratic mean.

Motivation

Using a quadratic mean could be an interesting alternative to a Polynomial kernel of degree two, reducing the risk getting less singular matrixes when fitting a GP model to some rough training datasets by considering simpler kernels conjointly with this mean module.

Checklist

[x] Add the gradient and the hessian
[x] Allow linear operations (esp. Quadratic + Linear)
[x] Add unit tests
[x] Give examples in 1D
[ ] Check if the documentation is well generated

I have run pip install -e . but I cannot generate locally the docs. Do you have any idea on how to install all the dependencies correctly for sphinx?

Ideas for further PRs

Reduce the number of files inside the means folder, e.g. gather *.py, *_grad.py and *_gradgrad.py into a single file.
Refactor the unit tests for means as the base test for means is always re-implemented. We could think of a way to shorten the boiler plate code.