JuliaPOMDP / POMDPs.jl

MDPs and POMDPs in Julia - An interface for defining, solving, and simulating fully and partially observable Markov decision processes on discrete and continuous spaces.
http://juliapomdp.github.io/POMDPs.jl/latest/
Other
652 stars 99 forks source link

TMaze + DiscreteUpdater errors #545

Closed FlyingWorkshop closed 4 weeks ago

FlyingWorkshop commented 3 months ago

Steps to recreate:

using POMDPs, POMDPTools, POMDPModels

pomdp = TMaze(1, 0.99)
updater = DiscreteUpdater(pomdp)
b0 = DiscreteBelief(pomdp, [1.0, 0.0, 0.0, 0.0, 0.0])
a = 1
o = 2
b = update(updater, b0, a, o)

Error:

ERROR: LoadError: Failed discrete belief update: new probabilities sum to zero.

b = DiscreteBelief{TMaze, Union{TerminalState, TMazeState}}(TMaze(1, 0.99), Union{TerminalState, TMazeState}[TMazeState(1, :north), TMazeState(1, :south), TMazeState(2, :north), TMazeState(2, :south), TerminalState()], [1.0, 0.0, 0.0, 0.0, 0.0])
a = 1
o = 2

Failed discrete belief update: new probabilities sum to zero.

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] update(bu::DiscreteUpdater{TMaze}, b::DiscreteBelief{TMaze, Union{TerminalState, TMazeState}}, a::Int64, o::Int64)
   @ POMDPTools.BeliefUpdaters ~/.julia/packages/POMDPTools/7Rekv/src/BeliefUpdaters/discrete.jl:133
 [3] top-level scope
   @ ~/VSCodeProjects/BeliefCompression/arena/tmaze_benchmark.jl:8
in expression starting at /Users/logan/VSCodeProjects/BeliefCompression/arena/tmaze_benchmark.jl:8

Is this expected? You can also trigger the error with this:

using POMDPs, POMDPTools, POMDPModels
using DiscreteValueIteration

pomdp = TMaze(1, 0.99)
updater = DiscreteUpdater(pomdp)
b = initialize_belief(updater, initialstate(pomdp))
for _ in 1:10
    for a in actions(pomdp, b)
        for o in observations(pomdp)
            b_new = update(updater, b, a, o)
            global b = b_new
        end
    end
end
mykelk commented 3 months ago

I think this is because you can't observe 2 when you are in state 1. Since b0 assigns probability 1 to being in state 1, then this will fail as expected.