Open simsurace opened 2 years ago
I think that for quick and dirty likelihood creations, it would be more practical to have a CustomLikelihood
:
struct CustomLikelihood{F} <: AbstractLikelihood
f_to_dist::F
end
# For example to create a Binomial distribution with logistic invlink
l = CustomLikelihood(f->Binomial(10, f))
And for other likelihoods, users can always create their own type
struct MyLikelihood{L} <: GPLikelihoods.AbstractLikelihood
invlink::L
end
and do whatever they want with it.
The interest of having different subtypes is that one can easily multiple-dispatch on them. What you proposed with the aliases would work but be a bit lengthy ::GPLikelihood{<:Bernoulli,<:LogisticLink}
instead of ::Bernoulli{<:LogisticLink}
.
See for example here: https://github.com/theogf/AugmentedGaussianProcesses.jl/blob/eb5eb7f886c209a80e596ea6e0f678dd2e5f0a7b/src/likelihood/logistic.jl#L60
Ah, I screwed up the const
definitions, they should be on the level of types. To do what you want with dispatch using the GPLikelihood
type, one could define special cases as follows:
const BernoulliLikelihood{T} = GPLikelihood{<:Bernoulli, T} where T
etc.
Apart from the question of what is the best way to construct likelihoods, the issue here (I think) is whether it would be possible to have fallbacks for some or most of the methods that are used by the other packages in the ecosystem. Is there already a consensus on what would be the standard API for subtypes of AbstractLikelihoods? Things like loglikelihood
could be directly taken from Distributions, and gradients could be calculated with some autodiff package, right? In case that custom methods provide better performance, they can always be defined later.
That's another good point. We never really set on a proper API. This issue could be a good start for a discussion.
The general idea so far is that any AbstractLikelihood
applied on f
(or fs
) will return a Distribution
.
We could provide some nice wrapping like
Distributions.loglikelihood(l::AbstractLikelihood, y, f) = loglikelihood(l(f), y)
but it would be nice to generally not be too opinionated about the choices we make.
Another thing planned is to move some expectation computation tools from ApproximateGPs
here.
To get a better understanding of all the moving pieces, it'd be really good to have some concrete examples (in code) of likelihoods both with additional parameters and multiple latent functions (e.g. Normal with trainable but scalar variance, Normal with both mean and variance parameterised by GPs...)
Following issue #231 in AbstractGPs.jl, I would like to propose an interface that allows to automatically use Distributions.jl's large library of distributions. Something like (I haven't tested this yet):
and then have convenience structs for special cases:
Would this be useful?