Add generalized parallel-scan algorithms for dynamic factor graphs

[eb8680]

Generalizing tensor variable elimination to dynamic factor graphs

• [x] #398 A version of modified_partial_sum_product that works for time lag 1, and tests that compare it to partial_sum_product applied to the unrolled factors.
• [x] #400 Making modified_partial_sum_product use MarkovProduct and additional tests that verify that it works with funsor.optimize.apply_optimizer
• [ ] A version of modified_partial_sum_product that uses sarkka_bilmes_product directly, and tests that compare it to partial_sum_product applied to unrolled factors
• [ ] Tests verifying that the sarkka_bilmes_product-based modified_partial_sum_product works with funsor.optimize.apply_optimizer, and any fixes necessary to make them pass (e.g. a first-class funsor.Funsor wrapper for sarkka_bilmes_product analogous to MarkovProduct)
• [ ] A nice Funsor example or two to show off the new algorithm

Optional or out of scope:

• [ ] Optional: Reformulate algorithm in terms of recursive rewrite rules
• [ ] Optional: removal of less general algorithms implemented in previous steps
• [ ] Optional: Integration with Pyro for use with enumeration

Parallel-scan operation implementation

• [x] #292 Implement parallel-scan sarkka_bilmes_product operation
• [x] #294 Support global variables by passing a global_vars argument
• [x] #294 Add Gaussian test cases
• [x] #299 Add partially parallel elimination orders
• [x] #409 Handle partial windows, either by padding or by unrolling the first chunk
• [x] #415 Change sarkka_bilmes_product dimension name prefix from P to something less likely to interfere with user code
• [ ] Make sarkka_bilmes_product a first-class Funsor analogous to MarkovProduct
• [ ] Add docs defining the operation and explaining the parallelism-memory tradeoff for both naive_sarkka_bilmes_product and sarkka_bilmes_product
• [ ] Add some examples illustrating the use of sarkka_bilmes_product

Optional tasks:

• [ ] Optional: Add moment_matching tests
• [ ] Optional: Update sarkka_bilmes_product interface to follow the subscripting design in #167 or the MarkovProduct-like design in #399
• [ ] Optional: Change function name sarkka_bilmes_product to something more descriptive
• [ ] Optional: reformulate sequential_sum_product into recursive rewrite rule

[eb8680]

@ordabayevy I copied the work plan from #398 to this issue. I suggest going ahead with making modified_partial_sum_product use the first-class Funsor funsor.sum_product.MarkovProduct in place of sequential_sum_product and testing that the result is fully compatible with funsor.optimizer.apply_optimizer, since I think that may be necessary for your problem, but the other stuff is less important.

[ordabayevy]

@eb8680 sounds good. Can you point to an example that tests compatibility with funsor.optimizer.apply_optimizer?

[eb8680]

Can you point to an example that tests compatibility with funsor.optimizer.apply_optimizer?

See my PR #400 - that should address this point.