Trouble working contrast ratios into hierarchical model

[cmlamman]

Vinny, Solomon,

I have set up a hierarchical model to include a fit for mass ratios based on the contrast ratio data in addition to just separations. The notebook for this is attached.

Although it runs for the first steps, I keep getting the error shown in the attached notebook. I think the issue is in how I'm setting my priors, and that the problem is either in line 20, 33, or 48 of the 5th cell. Any advice would be greatly appreciated!

hrarcl_pymc3_model.zip

[vnmanoharan]

My guess is that the sampler is going outside the bounds, and that's why it's getting derivatives equal to zero. This probably has something to do with the way your model is set up. I would first try it without the mass ratio to contrast ratio mapping, assume that the mass ratios are directly observed, and see if the sampler throws any errors.

The problem might lie with all the bounded variables you set up at the beginning. You could try using pm.Pareto instead of pm.Kumaraswamy, and inverting your mass ratios so that they go from 1 to infinity. That way you don't have to bound the distribution: you just set the parameter m in the Pareto distribution to 1 (I think).

[vnmanoharan]

Also you can use a Gamma prior for your power law index rather than a bounded normal. The Gamma distribution has support (0,\infty), and so you wouldn't have to call pm.Bound. This would get rid of another bound.

[cmlamman]

Thanks Vinny, I implemented both of these changes. When I do not integrate the samples that fall outside observation limits from the likelihood (by commenting out lines 62-80), the fit runs ~80% of the time without throwing the Mass matrix contains zeros on the diagonal error.

When I do try accounting for observation limits, (specifically lines 68, 79, 80) - I always get that error. I have been trying a few different priors (line 68), but I'm worried that I'm using the right_censored_likelihood function wrong here, since the cutoff I'm inputting is dependent on separations, not just one sharp cutoff, as is used in lines 75-76 for angular separations. If this is the case, I don't know where to start on modeling that cutoff.

hrarcl_pymc3_model1.zip

[vnmanoharan]

It looks like the censored likelihood functions aren't the right ones to use here. You actually want the truncated likelihood, which is a little simpler. I think you just need to change your likelihood function from pm.Normal to pm.TruncatedNormal. I'm still not sure how to deal with marginalizing over the number of unobserved data points, but you might start by just using the truncated distribution and then we can figure out how to add in the number of unobserved points.

[vnmanoharan]

Also, it looks like you can use TruncatedNormal for your separation distance prior, if you don't like the Gamma prior we discussed earlier today.

[cmlamman]

I got the fit to consistently work with separations, contrast ratios, and accounting for truncating in both! (at least the 10 times I ran it)

Your suggestions helped a lot, and another fix that made a difference was to actually fit the inverse contrast ratios - before I was just modeling the inverse mass ratios then converting them to normal contrast ratios.

Unfortunately the fit is pretty bad, but I'll open another issue for that.

Thanks Vinny!!