blei-lab / edward

A probabilistic programming language in TensorFlow. Deep generative models, variational inference.
http://edwardlib.org
Other
4.84k stars 759 forks source link

Edward Roadmap #464

Open dustinvtran opened 7 years ago

dustinvtran commented 7 years ago

Following the 2017 TensorFlow Dev Summit, here is an outline of Edward going forward at least for Spring 2017. Of course, comments are always welcome. I'm happy to change priorities subject to interest. Related issues are added in parentheses.

Here are features I probably don't have time for. However, they are very much on my mind, and are impactful additions if someone wants to take the helm. (Any of the above is also appreciated!)

PKU-YYang commented 7 years ago

Hi Dustin:

Just wondering if there is any Control variates (#4) tricks already implemented in Edward? I noticed there is a branch called rao-blakcwellization, not sure if it works properly now.

Thanks.

dustinvtran commented 7 years ago

Not at the moment. I was working out RB as you pointed out, which turns out to be quite involved. Control variates are much easier to implement; I haven't looked into it quite yet.

ip01 commented 7 years ago

Hi Dustin, do you, or perhaps Francisco, or whomever, plan to implement Generalized Reparametrization Gradient in Edward? If yes - could it be usefull as effective black-box technique for large-scale topic models (Dirichlet G-REP'ed via exp-covariance, Multinomial via Gumbel-Softmax)?

dustinvtran commented 7 years ago

I've never experimented with it so I'm not sure. pinging @franrruiz. I do think there's no one-size-fits-all solution—especially for discrete random variables, and whether it be score functions+control variates, g-rep, gumbel softmax, or reparameterization gradients. However, it would be certainly useful to have them all to experiment.

RoyiAvital commented 4 years ago

Is anyone keep working on this?