Unify Shogun's probability distributions

[karlnapf]

This is an entrance task for the MCMC GSoC 2014 project http://www.shogun-toolbox.org/page/Events/gsoc2014_ideas#mcmc

Currently, Shogun has various ways to represent probability distributions: The latest addition was http://www.shogun-toolbox.org/doc/en/3.0.0/classshogun_1_1CProbabilityDistribution.html with implementation http://www.shogun-toolbox.org/doc/en/3.0.0/classshogun_1_1CGaussianDistribution.html

There is an old one (which has a slightly different interface) http://www.shogun-toolbox.org/doc/en/3.0.0/classshogun_1_1CDistribution.html with implementation http://www.shogun-toolbox.org/doc/en/3.0.0/classshogun_1_1CGaussian.html

The latter two are old Shogun classes and reflect an old-school ML-type view on distributions. We want to consider renaming those (or move to new interface). The first two are recent additions, and more in the line of the MCMC project.

This task is about unifying and extending the existing distributions.

• Have a proper (STAN inspired) class hierarchy: Univariate, multivariate, both discrete and continuous
• actual implementations of those. The current Gaussian one can be re-used (in fact, this should be available even if stan is not installed). For others, we can rely on stan since they implement almost everything we will ever need, see https://github.com/stan-dev/stan/tree/develop/src/stan/prob
• Here is a minimal example to interface distributions in stan: https://gist.github.com/karlnapf/9354664
• This task depends also depends on #1875 since it requires a working stan
• Note that some distributions should also be available if stan is not present.
• There should be a distribution of discrete mixtures (possibly interfering/changing CGMM which is a mixture of Gaussian distribution)

[karlnapf]

And please talk to us before starting to work on this.

Some more comments:

• Sampling random numbers from distributions should rely on what @vigsterkr says here, i.e. c++ but not boost or stan
• stan/boost should be used for log_pdf or cdf methods
• For univariate distributions, we can in fact just rely on boost. Stan only for those complicated multivariate ones, but the class hierarchy and interfaces are independent of that anyways...

[vigsterkr]

one who is considering to work on this should really check out the new Pseudo-random number generation with c++11: http://en.cppreference.com/w/cpp/numeric/random

[karlnapf]

Another aspect is to be able to exploit stans automatic differentiation framework: It can automatically compute derivative for all distributions. This is done via https://en.wikipedia.org/wiki/Automatic_differentiation

[karlnapf]

Stan currently is kind of out-of-the-game due to its non-existing packaging structure. But we might get back to this. Still the task of representing probability distributions properly remains. Mostly interface work