The following are some notes on PMMH, which is an inference algorithm that it might be interesting to implement using the asynchronous FRP tools of Rhine. In particular, this would be an online version of the PMMH algorithm.
I use a >--> b to mean forall m . MonadDistribution m => BehaviourF m Double a b
The setting is that we have a transition model tm :: Theta >--> Z, that expresses how parameters theta influence hidden states, and an observation model om :: Z >--> Y which expresses how hidden states result in observations. Moreover, we have a prior pr :: () >--> Theta = constM pTheta, for some distribution pTheta. That is, theta is stationary. The naive approach of
particleFilter proc data -> do
theta <- pr -< ()
latent <- tm -< theta
arrM factor -< the_likelihood latent data
returnA -< (latent, theta)
will be, while asymptotically correct, useless, because the values of theta for all particles will never change from their initial state.
What we would like to do instead is run an MCMC chain over theta. To do this exactly, we would need to calculate $p(y,\theta)$ and use it in the MH ratio. However, (and this is the PMMH approach), we can instead use an unbiased estimate of $p(y,\theta)$ obtained by importance sampling or a particle filter. In particular, the output of system evidence as defined here:
evidence :: Theta >--> Double
evidence = sum . arr sumWeights . particleFilter proc theta -> do
latent <- tm -< theta
arrM factor -< the_likelihood latent data
is (roughly) what we need.
On the other hand, MCMC can be expressed as a (discrete) system:
mcmc :: Double >--> Theta
mcmc = ...
The general idea is that these would be run on separate clocks, but with communication. Every so often, the clock for mcmc would tick, at which point a new value of Theta would be chosen. Then evidence would run for a while, until a reasonably good estimate of the evidence was calculated, good enough to take another step of mcmc.
The following are some notes on PMMH, which is an inference algorithm that it might be interesting to implement using the asynchronous FRP tools of Rhine. In particular, this would be an online version of the PMMH algorithm.
I use
a >--> bto meanforall m . MonadDistribution m => BehaviourF m Double a bThe setting is that we have a transition model
tm :: Theta >--> Z, that expresses how parametersthetainfluence hidden states, and an observation modelom :: Z >--> Ywhich expresses how hidden states result in observations. Moreover, we have a priorpr :: () >--> Theta = constM pTheta, for some distributionpTheta. That is,thetais stationary. The naive approach ofwill be, while asymptotically correct, useless, because the values of theta for all particles will never change from their initial state.
What we would like to do instead is run an MCMC chain over theta. To do this exactly, we would need to calculate $p(y,\theta)$ and use it in the MH ratio. However, (and this is the PMMH approach), we can instead use an unbiased estimate of $p(y,\theta)$ obtained by importance sampling or a particle filter. In particular, the output of system
evidenceas defined here:is (roughly) what we need.
On the other hand, MCMC can be expressed as a (discrete) system:
The general idea is that these would be run on separate clocks, but with communication. Every so often, the clock for
mcmcwould tick, at which point a new value ofThetawould be chosen. Thenevidencewould run for a while, until a reasonably good estimate of the evidence was calculated, good enough to take another step ofmcmc.Obviously this is just a very high level sketch.