test fails for Claret-Bruno model

[gravesti]

Currently test to recover parameters in the unscaled variance model fails.

• I found found that increasing the sample size allows the parameters to be recovered, but this is very slow to fit.
• Warning: 4000 of 4000 (100.0%) transitions hit the maximum treedepth limit of 10.
• ess_bulk test fails for mu_c
• The found parameters are not completely bad.
• I am thinking whether posterior predictive checks could be used to determine if the mean parameters from the "bad" fit are actually quite a good fit for the data.

[gowerc]

Interesting...

I'm assuming that this implies the model works fine in the scaled case.

If memory serves me right ess_bulk failing means that they are very few independent samples and although the point estimates may be fine it means the overall coverage / shape of the posterior is likely not to be well described. Also means any variance estimates are likely to be inaccurate. I'm sure I read in one of the Stan articles that ESS > 100 is a minimum standard.

Did you look into the code at all ? Wondering if I made an implementation mistake ? If not I am wondering if maybe this model just isn't recoverable and we should instead just limit it to the scaled version only ?

I'm also assuming this is a duplicate of #407 ?

[gravesti]

Yes, scaled seems to work fine. Indeed, something is not good about the sampling of this model.

I started to look at the code, but didn't finish looking thoroughly.

Limiting to the scaled version seems like a shortcut to resolving this.

To my last point I was thinking we could implement pp_check(https://mc-stan.org/bayesplot/reference/pp_check.html). I with with the Quantities functionality, you have everything that's needed.