Reinstate and improve Search inference

eb8680 commented 6 years ago

pyro.infer.Search was removed in #1044 for performance and maintenance reasons, but it's being used informally in multiple places in Pyro and other projects, including in the RSA examples (#1198 ) and in contrib.oed (#1224 ). We should collect the various copies into one place and add broadcasting-based parallel search as well, since most of the applications have static structure.

Tasks:

[ ] Add dependent enumeration (#915) -based Search that creates a sequential TracePosterior
[ ] Extend Search to support mixed parallel and sequential enumeration, as in TraceEnum_ELBO
[ ] Move HashingMarginal from examples/rsa/search_inference.py to pyro.infer (proposed in #845 and finished in #1198)

fritzo commented 6 years ago

@eb8680 it would be nice to implement message passing for Search next week, at the same time we implement message passing for SVI and HMC. Is there any preparation we can do this week to set us up to implement the harder parts next week?

eb8680 commented 6 years ago

One big blocker is actually not having a batched EmpiricalMarginal/HashingMarginal, but I think that would be a bit of a pain to implement as well since Empirical isn't vectorized.

neerajprad commented 6 years ago

One big blocker is actually not having a batched EmpiricalMarginal/HashingMarginal, but I think that would be a bit of a pain to implement as well since Empirical isn't vectorized.

What do you think we need to vectorize Empirical - can we generate bootstrap samples off of some initial samples to fill arbitrary batches on the left, or do you foresee needing something more sophisticated?

pyro-ppl / pyro

Reinstate and improve Search inference #1226