Action in the loop

[reubenharry]

One of the exciting things about real-time inference is that you could take actions depending on your current belief state that might effect future observations.

For example, suppose you were modelling an agent looking for something on a 2D map. The direction it heads in might be a function of its belief about the position of that object, but its sensor data might depend on the direction it heads in. So, a control problem, I guess.

I wonder how easy it would be to target examples like this.

[reubenharry]

Recursive arrow notation seems like the right direction here...:

example :: IO ()
example = sampleIO $ reactimate proc () -> do 
    state <- prior -< ()
    rec 
        observation <- iPre (initialState, initialAction) >>> generativeModel -< (state, action)
        samples <- onlineSMC' 200 resampleMultinomial posterior -< observation
        action <- someFunctionOfSamples -< samples
    arrM (liftIO . print) -< bs

[turion]

I'd recommend feedback over rec + iPre in this example. It's too easy to use rec accidentally without iPre which can lead to unresolvable loops.

The direction it heads in might be a function of its belief about the position of that object, but its sensor data might depend on the direction it heads in. So, a control problem, I guess.

In principle, dunai-bayes and rhine-bayes should be good for control problems. It's hard to find the way out of the loop sometimes. But you have a natural delay in the real system when sending an action, because it will take some time to transmit your command and act according to it. So you only need to send the action through the delay & recursion, not the state.