Open gcgibson opened 7 years ago
It may be easier to just write your own sampler rather than putting a lot of sampler logic in the Stochastic/Logical node definitions.
For example see ## User-Defined Samplers
in http://mambajl.readthedocs.io/en/latest/tutorial.html or the example samplers in http://mambajl.readthedocs.io/en/latest/examples/pollution.html
@bdeonovic That makes sense, but the custom samplers still require some return
Gibbs_sigma2 = Sampler([:sigma2],
(mu, sigma2, y) ->
begin
a = length(y) / 2.0 + shape(sigma2.distr)
b = sumabs2(y - mu) / 2.0 + scale(sigma2.distr)
rand(InverseGamma(a, b))
end
)
In this case a random draw from InvGamma. In the case of a non positive definite sample I would like to simple reject it from the chain, can I hack my sampler to do that?
EDIT: To clarify, if I wanted to put an if statement like so
Gibbs_sigma2 = Sampler([:sigma2],
(mu, sigma2, y) ->
begin
a = length(y) / 2.0 + shape(sigma2.distr)
b = sumabs2(y - mu) / 2.0 + scale(sigma2.distr)
if matrix formed from some combination of a and b is positive semi definite:
rand(InverseGamma(a, b))
else
not sure what to put here
end
)
You might want to check why non-positive definite matrices are being formed. Sometimes a matrix is not positive definite in julia due to round off errors. In those cases you can explicitly specify to julia that the matrix is Positive definite.
Otherwise, you can always return the current iterate if the proposed matrix is not positive definite.
How do you get access to the underlying chain to return the last accepted step, I guess you don't need it you can just return the input to your sampler?
in the example you have
Gibbs_sigma2 = Sampler([:sigma2],
(mu, sigma2, y) ->
begin
##sigma2.value is the current value of sigma2 which you are trying to sample a new value for
a = length(y) / 2.0 + shape(sigma2.distr)
b = sumabs2(y - mu) / 2.0 + scale(sigma2.distr)
if matrix formed from some combination of a and b is positive semi definite:
rand(InverseGamma(a, b))
else
not sure what to put here
end
)
``
Awesome! Thanks for your help! The matrices are PSD until s^2 gets sufficiently small
Unfortunately, I can't use Gibbs sampling because the conditional density is not available (unlike Bayesian linear regression , gaussian processes don't have an analytical distribution for s2 that I know of ). Is there anyway to get a code snippet like that into MH or NUTS?
Your custom sampler doesn't have to be a Gibbs sampler. It can be whatever you want. For MH you would just need to calculate the the acceptance probability for example.
But then I would need a proposal dist etc right? I would basically have to re-implement MH in that code block? Gibbs seemed nice because it was just actually sampling from known dists, but I would prefer to pass a flag to the MH sampler saying treat this exception as probability 0, if possible.
I have written the following code for fitting a simple GP
which throws a
while exploring the posterior for certain samples. Is there anyway to reject a sample based on this exception and keep going? I am not sure that is the correct way of handling bad s^2 values so if anyone has other suggestions for what to do please let me know!