Conjugate sampler or Conjugate distribution

[nabriis]

This following issue springs from discussions had in #31. In particular the comments from jakj:

I am unsure about the idea of a single (approximate) conjugate sampler class. For two reasons I think. First, will this not amount to essentially a long if-else covering all the supported, and more or less unrelated, pairs? As such would seem splitting in dedicated classes might make sense? Second, and I am unsure here, it seems that from a choice of a pair of DISTRIBUTIONS, we are choosing a conjugate SAMPLER, whereas it would seem more intuitive to be selecting conjugate DISTRIBUTIONS (though of course I do realize that the place in which this currently happens is where a name of a sampler is expected).

• [ ] Update Gibbs tutorial with better explanation of Conjugate relations

[amal-ghamdi]

one possibility is to use static methods (similar to testproblem get component method).

To have for example:

x = Gamma.as_conjugate(target)
y = Gamma.as_approx_conjugate(target)

Where the static method constructs Gamma parameters from the target and returns Gamma distribution object.

[nabriis]

One of the main areas where this would be used is in the Gibbs sampler. Currently the Gibbs sampler will determine sampling strategy by conditioning on all other variables and from this detecing the type of Posterior.

I think that a check for conjugacy should likely belong to Posterior somehow. Making it detect if the two defined distributions are conjugate, and then essentially converting to a Gamma distribution (in the Gauss-Gamma case).

In many ways I think the Posterior class could be where we would want to handle specific automatic choices of sampler (maybe as multiple classes). This would men that Gibbs could just always call target.step to do a single MCMC step. If a Distribution, step produces one sample, if a Posterior step produces one step in MCMC chain.

This would also mean the user could define a Posterior and actually call .sample to sample from it. It makes it more intuitive like that. Further, BayesianProblem is using JointDistribution and Posterior anyway, so the functionality would just be refactored from there easily.