search

blei-lab / edward

A probabilistic programming language in TensorFlow. Deep generative models, variational inference.
http://edwardlib.org
Other
4.83k stars 760 forks source link

truncated DP mixture model example #183

Open dustinvtran opened 7 years ago

dustinvtran commented 7 years ago

(Elham Azizi asks about DP mixture models for use in some of her work with Dana Pe’er in computational biology.)

part of my reply below:

DPs in their full infiniteness aren’t possible in Edward for similar reasons as Stan. It requires lazy evaluation to bake in a generative process with an infinite number of latent variables but whose output only uses a finite number of latent variables. We’re still working on that regard.

Truncated DP mixture models are definitely possible, e.g., setting the maximum number of stick breaks a priori. And it seems like that’s what your work uses. I imagine Edward is the fastest for inferring it in practice because all other systems out there use MCMC or SMC. And Edward will handle data subsampling and things like GPU support or distributed training if you want to take advantage of them.

Would be great if someone gets a truncated DP mixture model up and running!

AustinRochford commented 7 years ago

I have a PyMC3 truncated DP mixture model example currently via MCMC here. It may be possible to adapt this into an Edward example by swapping out some of the Theano-specific bits, if necessary. I'd be happy to give this a shot once I get back from an internet-less vacation next week.

dustinvtran commented 7 years ago

@AustinRochford: yeah that would be fantastic if you could.

elhamazizi commented 7 years ago

That would be really helpful, thanks! @AustinRochford

AlexPiche commented 7 years ago

@AustinRochford, I was wondering if you were working on the issue? If yes, I would be happy to help you out. I am familiar with DP, nested DP and hierarchical DP mixtures.

AustinRochford commented 7 years ago

@AlexPiche I had done some work on it, then my day job got in the way. I am hoping to pick it up again in the next two or three weeks

jengelman commented 7 years ago

@AustinRochford did you ever end up picking this up again? I'm doing some work on nonparametrics and would be happy to contribute.

AustinRochford commented 7 years ago

I did not get a chance to get very far on this, so feel free to pick it up if you're so inclined!

elhamazizi commented 6 years ago

Hi Dustin,

Hope all is well! I was wondering if you know if anyone ever implemented the DP mixture model in Edward. We have another (comp bio) model that we'd like to implement in Edward if possible.

Thanks!

Best, Elham

On Thu, Apr 6, 2017 at 12:47 PM Joshua Engelman notifications@github.com wrote:

@AustinRochford https://github.com/AustinRochford did you ever end up picking this up again? I'm doing some work on nonparametrics and would be happy to contribute.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/blei-lab/edward/issues/183#issuecomment-292234801, or mute the thread https://github.com/notifications/unsubscribe-auth/AHrctsuQhJbu6TDP1uyoOBUpZ2iwFuBMks5rtRczgaJpZM4JSJJt .

dustinvtran commented 6 years ago

hi @elhamazizi. The answer is an unfortunate no. I believe there's a truncated DP model if you search the Forum. But we never got efficient inference to work on it(?).

elhamazizi commented 6 years ago

Hi Dustin,

Thanks, the truncated DP seems to work for now. Another question we have is whether you're planning to incorporate the Inverse-Wishart distribution into Edward?

Best, Elham

On Tue, Feb 13, 2018 at 12:15 PM Dustin Tran notifications@github.com wrote:

hi @elhamazizi https://github.com/elhamazizi. The answer is an unfortunate no. I believe there's a truncated DP model if you search the Forum. But we never got efficient work on it(?).

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/blei-lab/edward/issues/183#issuecomment-365336922, or mute the thread https://github.com/notifications/unsubscribe-auth/AHrcth00OtomewELpmtX_D13hBtHN8Gtks5tUcNHgaJpZM4JSJJt .

dustinvtran commented 6 years ago

You can try submitting a feature request for that in TF's Github issues.

On Feb 16, 2018, at 10:17 AM, elhamazizi notifications@github.com wrote:

Hi Dustin,

Thanks, the truncated DP seems to work for now. Another question we have is whether you're planning to incorporate the Inverse-Wishart distribution into Edward?

Best, Elham

On Tue, Feb 13, 2018 at 12:15 PM Dustin Tran notifications@github.com wrote:

hi @elhamazizi https://github.com/elhamazizi. The answer is an unfortunate no. I believe there's a truncated DP model if you search the Forum. But we never got efficient work on it(?).

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/blei-lab/edward/issues/183#issuecomment-365336922, or mute the thread https://github.com/notifications/unsubscribe-auth/AHrcth00OtomewELpmtX_D13hBtHN8Gtks5tUcNHgaJpZM4JSJJt .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/blei-lab/edward/issues/183#issuecomment-366315803, or mute the thread https://github.com/notifications/unsubscribe-auth/ACc2i2PX7fjvll4DdYIQJqIrZurT0-S0ks5tVcYcgaJpZM4JSJJt.