search

charles-river-analytics / figaro

Figaro Programming Language and Core Libraries
Other
756 stars 152 forks source link

How to Substitute Inference Algorithm? (Building LDA through Figaro) #371

Closed windweller closed 9 years ago

windweller commented 9 years ago

In the scope of machine learning, there are many probability-based algorithms, especially Latent Dirichlet Allocation (LDA). There are four inference algorithms they use to calculate posterior Dirichlet: Mean Field Variational methods, Expectation propagation, Collapsed Gibbs Sampling, or Collapsed Variational inference. I am wondering if it's possible to build a complex model like LDA through Figaro. If not by the already-supported inference algorithm, how much can we add custom inference algorithmic support?

bruttenberg commented 9 years ago

Hi Allen,

Thanks for your interest in Figaro. We have explored trying to represent and reason about non-parametric versions of LDA in the past (Hierarchical Dirichlet Process). Representing LDA and other topic models is actually quite nice in Figaro, but since we don’t have any of the traditional methods for solving them (variational methods or gibbs sampling), inference is very difficult to do correctly.

Figaro is extensible and has APIs for people to build new algorithms. So given that representation of the model is easy in Figaro, building a custom algorithm to solve the topic model is definitely feasible.

Feel free to let us know if you want to discuss it more!

Brian

From: Allen Nie [mailto:notifications@github.com] Sent: Friday, January 16, 2015 11:57 AM To: p2t2/figaro Subject: [figaro] How to Substitute Inference Algorithm? (#371)

In the scope of machine learning, there are many probability-based algorithms, especially Latent Dirichlet Allocation (LDA). There are four inference algorithms they use to calculate posterior Dirichlet: Mean Field Variational methods, Expectation propagation, Collapsed Gibbs Sampling, or Collapsed Variational inference. I am wondering if it's possible to build a complex model like LDA through Figaro. If not by the already-supported inference algorithm, how much can we add custom inference algorithmic support?

— Reply to this email directly or view it on GitHubhttps://github.com/p2t2/figaro/issues/371.

apfeffer commented 9 years ago

Hi Allen,

Let me just add to Brian’s response that we’re very interested in new algorithms for Figaro, and we’d be happy to provide you with assistance or guidance if you want to build something.

Avi

From: bruttenberg [mailto:notifications@github.com] Sent: Friday, January 16, 2015 1:47 PM To: p2t2/figaro Subject: Re: [figaro] How to Substitute Inference Algorithm? (Building LDA through Figaro) (#371)

Hi Allen,

Thanks for your interest in Figaro. We have explored trying to represent and reason about non-parametric versions of LDA in the past (Hierarchical Dirichlet Process). Representing LDA and other topic models is actually quite nice in Figaro, but since we don’t have any of the traditional methods for solving them (variational methods or gibbs sampling), inference is very difficult to do correctly.

Figaro is extensible and has APIs for people to build new algorithms. So given that representation of the model is easy in Figaro, building a custom algorithm to solve the topic model is definitely feasible.

Feel free to let us know if you want to discuss it more!

Brian

From: Allen Nie [mailto:notifications@github.com] Sent: Friday, January 16, 2015 11:57 AM To: p2t2/figaro Subject: [figaro] How to Substitute Inference Algorithm? (#371)

In the scope of machine learning, there are many probability-based algorithms, especially Latent Dirichlet Allocation (LDA). There are four inference algorithms they use to calculate posterior Dirichlet: Mean Field Variational methods, Expectation propagation, Collapsed Gibbs Sampling, or Collapsed Variational inference. I am wondering if it's possible to build a complex model like LDA through Figaro. If not by the already-supported inference algorithm, how much can we add custom inference algorithmic support?

— Reply to this email directly or view it on GitHubhttps://github.com/p2t2/figaro/issues/371.

— Reply to this email directly or view it on GitHubhttps://github.com/p2t2/figaro/issues/371#issuecomment-70301607.

windweller commented 9 years ago

I thought Figaro's ability to build model is quite unmatched, which reminds me of MatLab or R. I'll explore more on Figaro and its mechanism, including the possibility of adding Gibbs sampling or variational methods. Thanks for the quick response.