I came across this cute example of nested semiring dynamic programming in the context of Bayesian optimization. Thompson sampling first performs Bayesian regression, fitting a posterior p(θ|Xs,ys) over parameters θ given a fully-supervised dataset of (X,y) pairs, then repeatedly samples parameters θ and optimizes expected reward E[y|X,θ] over X. That is, at each step
X_optimal <- arg max_X int_y int_θ y p(y|θ,X) p(θ) prod_i p(yi|θ,Xi)
# new choice --reward-- prior ---likelihood---
For example in Pyroedp(θ) prod_i p(yi|θ,Xi) is approximated variationally, and E[y|θ,X] is just a tensor contraction (although there it is optimized via annealed Gibbs sampling rather than via dynamic programming, as an aside we could add an annealed Gibbs sampling interpretation to Funsor for sum-product contractions).
What would be a good example here, where we might leverage Funsor's dynamic programming to implement both the integral and argmax operators?
I came across this cute example of nested semiring dynamic programming in the context of Bayesian optimization. Thompson sampling first performs Bayesian regression, fitting a posterior
p(θ|Xs,ys)over parametersθgiven a fully-supervised dataset of(X,y)pairs, then repeatedly samples parametersθand optimizes expected rewardE[y|X,θ]overX. That is, at each stepFor example in Pyroed
p(θ) prod_i p(yi|θ,Xi)is approximated variationally, andE[y|θ,X]is just a tensor contraction (although there it is optimized via annealed Gibbs sampling rather than via dynamic programming, as an aside we could add an annealed Gibbs sampling interpretation to Funsor for sum-product contractions).What would be a good example here, where we might leverage Funsor's dynamic programming to implement both the integral and argmax operators?