search

JuliaGaussianProcesses / KernelFunctions.jl

Julia package for kernel functions for machine learning
https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/
MIT License
267 stars 32 forks source link

Wct/linear spline #475

Closed willtebbutt closed 1 year ago

willtebbutt commented 1 year ago

Summary

Found this interesting kernel in the Probabilistic numerics textbook.

Proposed changes

Add the LinearSplineKernel

What alternatives have you considered?

Doing more work to find a more general version of the spline kernel.

Breaking changes

None

@theogf does our AD correctness testing infrastructure currently permit me to test with Vector{Float64} inputs to kernelmatrix etc, rather than Matrix{Float64}? If not, I'll not bother testing. If it does, I'll add a line.

devmotion commented 1 year ago

Found this interesting kernel in the Probabilistic numerics textbook.

Haha, I just started to read through that book a bit some weeks ago :smile:

I'm not sure though if we should restrict "linear spline kernel" to this specific example. It seems a quick Google lookup yields many publications with different "linear spline kernels" (e.g., https://www.diva-portal.org/smash/get/diva2:716642/FULLTEXT01.pdf, https://people.vcu.edu/~ymotai/Publications/journals/TNNLS2015.pdf, https://gpy.readthedocs.io/en/deploy/_modules/GPy/kern/src/spline.html). Maybe we could generalize the definition to accommodate for (some of) these other forms as well?

Could you also add it to the documentation?

willtebbutt commented 1 year ago

Good point. It looks like the Spline kernel in GPy is what Hennig refers to as the cubic spline kernel in his talk.

The other publications seem to have quite different kernels, so I'm inclined to just reference the same things as GPy and be done with it. I'll add in the cubic spline as a separate kernel, as it seems to be quite different from the linear spline.

willtebbutt commented 1 year ago

Hmmmm I'm going to have to leave this open for a while. I don't currently have access to Wahba's 1980 paper in which I assume the stuff that Hennig talks about is discussed in more depth. At the minute, I'm not able to determine what the bounds are on the domain of the kernels. If anyone has access and can figure it out, I would greatly appreciate it.

st-- commented 1 year ago

@willtebbutt do you mean this one: Spline bases, regularization, and generalized cross validation for solving approximation problems with large quantities of noisy data ? Or the 1990 Spline models for observational data?

willtebbutt commented 1 year ago

I'm not going to find time to finish this off, so I'm going to close.