search

pyro-ppl / pyro

Deep universal probabilistic programming with Python and PyTorch
http://pyro.ai
Apache License 2.0
8.6k stars 986 forks source link

Automated test generation for inference #18

Closed eb8680 closed 7 years ago

eb8680 commented 7 years ago

Along the lines of #16 we should follow webPPL's example closely and make the inference test interface more generic, automate test creation, and write test models independently of the tests themselves so that they can be reused. We could even reuse the test data/ground truths from webPPL and other projects where possible.

martinjankowiak commented 7 years ago

i'll take the lead on this once any blocks resulting from map_data/batching stuff is clarified

eb8680 commented 7 years ago

Hypothesis seems like a really cool library that could make these tests more powerful and robust

jpchen commented 7 years ago

While working on stateless distributions, I realized that the nonreparameterized tests (eg NormalNormal, LogNormalNormal are still running the parameterized versions. Subtle bug, but after fixing it, those tests are now failing.

Discussed with @eb8680 and decided it was best to remove them for now, since this is more an issue with the tests than the elbo.

@karalets noticed the same thing while playing with the cubo estimator.