search

shogun-toolbox / shogun

Shōgun
http://shogun-toolbox.org
BSD 3-Clause "New" or "Revised" License
3.03k stars 1.04k forks source link

port new kernel functions from other libraries #3139

Open yorkerlin opened 8 years ago

yorkerlin commented 8 years ago

@karlnapf

ManifoldKernel LocalLengthScalesKernel HeteroscedasticKernel at https://github.com/jmetzen/gp_extras

yorkerlin commented 8 years ago

ExpSineSquared Kernel http://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.kernels.ExpSineSquared.html WhiteKernel http://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.kernels.WhiteKernel.html Matern Kernel http://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.kernels.Matern.html

karlnapf commented 8 years ago

@yorkerlin please give your issues meaningful names and tag them (this is a good entrance task) I agree it would be cool to add these kernels

yorkerlin commented 8 years ago

it will be great if these kernels can be added to Shogun using openMP and Shogun's linalg lib

souravsingh commented 8 years ago

I would like to work on the issue. How do I start?

yorkerlin commented 8 years ago

@souravsingh (1) The first step is to be familiar with these kernels at https://github.com/jmetzen/gp_extras and scikit-learn. It will be great if you know the math behind the implementation. For starters, you should focus on WhiteKernel or Matern Kernel or ExpSineSquared Kernel. BTW, Matern Kernel is a popular choice in kernel learning.
ManifoldKernel, LocalLengthScalesKernel and HeteroscedasticKernel are more complicated.

(2) The second step is to design an interface of these kernels for Shogun. You can take a look at http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CKernel.html If you are familiar with GaussianKernel (A.K.A. RBFKernel), you may take a look at http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CGaussianKernel.html http://www.shogun-toolbox.org/doc/en/latest/classshogun_1_1CGaussianARDKernel.html

You can also look at examples and unit tests for the usage of the kernel API. https://github.com/shogun-toolbox/shogun/tree/develop/tests/unit/kernel https://github.com/shogun-toolbox/shogun/blob/develop/examples/undocumented/libshogun/kernel_gaussian.cpp

(3) In shogun, we use the linalg lib for matrix computation. To be familiar with the lib, you may look at https://github.com/shogun-toolbox/shogun/blob/develop/src/shogun/kernel/ExponentialARDKernel.cpp https://github.com/shogun-toolbox/shogun/blob/develop/src/shogun/kernel/GaussianARDKernel.cpp You can find more usage of the linalg lib at https://github.com/shogun-toolbox/shogun/tree/develop/tests/unit/mathematics/linalg

@lambday could help you if you have any questions about the linalg lib.

(4) Finally, It will be great if you can implement any of these kernels, document the implementation for users and developers, do unit tests, and write notebooks/examples.

yorkerlin commented 8 years ago

Also take a look at https://github.com/shogun-toolbox/shogun/wiki/README_linalg

souravsingh commented 8 years ago

@yorkerlin Where will the kernels come under? Do we need to create a separate folder for the Gaussian Processes in case of Matern kernel or do I put it somewhere?

yorkerlin commented 8 years ago

@souravsingh All kernels should be in folder src/shogun/kernel. (e.g. GaussianARDKernel) One exception: if your implementation supports the sparse approximation (ref: http://www.jmlr.org/papers/volume6/quinonero-candela05a/quinonero-candela05a.pdf), folder src/shogun/machine/gp/ may be better. (e.g. GaussianARDSparseKernel)