search

pymc-devs / pymc

Bayesian Modeling and Probabilistic Programming in Python
https://docs.pymc.io/
Other
8.72k stars 2.02k forks source link

Chinese Restaurant Process #310

Closed malmaud closed 11 years ago

malmaud commented 11 years ago

I was wondering if there is support for the Chinese Restaurant Process so that I could write e.g. a Dirichlet process mixture model. It's not clear to me how to dynamically add and remove clusters to the model during inference in the pymc framework.

fonnesbeck commented 11 years ago

Though there is no formal DPP objects in PyMC, I give an example of a constructive stick-breaking DPP here. The probabilities generated in the stick-breaking process are then used in a categorical stochastic to form a non-parametric random effect. Is this what you had in mind?

malmaud commented 11 years ago

Yes, except without artificial truncating.

On Fri, Jul 12, 2013 at 2:38 PM, Chris Fonnesbeck notifications@github.comwrote:

Though there is no formal DPP objects in PyMC, I give an example of a constructive stick-breaking DPP herehttp://stronginference.com/post/implementing-dirichlet-processes-for-bayesian-semi-parametric-models. The probabilities generated in the stick-breaking process are then used in a categorical stochastic to form a non-parametric random effect. Is this what you had in mind?

— Reply to this email directly or view it on GitHubhttps://github.com/pymc-devs/pymc/issues/310#issuecomment-20905812 .

fonnesbeck commented 11 years ago

Do you mean truncating the number of groups? I usually just select a number of breaks that is almost as large as the number of observations. If the estimates from a preliminary run seem to push against that bound, I just make it larger.

malmaud commented 11 years ago

How efficient will inference be with a large number of groups, though? Won't there be a lot of rejections if it's proposing changing all the stick weights at once?

fonnesbeck commented 11 years ago

I've never had a problem with constructive DPP used this way. I tend to get convergence pretty quickly. You can help things along by using adaptive Metropolis on the betas.