How can we gain access to the updating Belief Vector?

[thecoldviews]

I am using the POMCP solver on a given problem, but I would like to visualize how the belief over the states gets modified at each time step. Is there some way I can guess access to the belief vector or some other method that helps with the same?

[zsunberg]

Hi @thecoldviews ,

Do you mean the outer loop belief of the agent, or the implicit beliefs at each node of the search tree? I'll assume that you mean the outer loop agent belief.

Beliefs can take a variety of forms in POMDPs.jl, so they are not always represented as a vector. For example, by default, POMCP uses a particle filter from ParticleFilters.jl. Most beliefs implement the distribution interface, so you can interact with them using the functions in this list: http://juliapomdp.github.io/POMDPs.jl/latest/interfaces/#distributions

For example here is a function that prints out the states and their probabilities in the belief

function show_belief(b)
    for s in iterator(b)
        @show s
        @show pdf(b, s)
    end
end

You could replace this with some more convenient way to plot the belief, etc.

If you have run a simulation and are left with a history hist, you can show each of the beliefs like this

for b in eachstep(hist, "b")
    show_belief(b)
end

Or if you want to see the belief as the simulation is running, you can do

for b in stepthrough(pomdp, planner, "b", max_steps=10)
    show_belief(b)
end

For more documentation on simulation and accessing results, see https://github.com/JuliaPOMDP/POMDPToolbox.jl#simulators

Hope that helps! Let me know id you have any other questions.

[thecoldviews]

Thank you! This was more than helpful!