search

JuliaPOMDP / POMDPs.jl

MDPs and POMDPs in Julia - An interface for defining, solving, and simulating fully and partially observable Markov decision processes on discrete and continuous spaces.
http://juliapomdp.github.io/POMDPs.jl/latest/
Other
652 stars 99 forks source link

Need to check if the belief is terminal in stepthrough #549

Closed himanshugupta1009 closed 1 month ago

himanshugupta1009 commented 1 month ago

I believe there needs to be an additional check in the Base.iterate function here when the iterator is of type POMDPSimIterator -

https://github.com/JuliaPOMDP/POMDPs.jl/blob/d837f1650c80787ce4ca6a5230813698b9eac9a6/lib/POMDPTools/src/Simulators/stepthrough.jl#L85

It is possible that the true state is not terminal, but the belief represented using a particle set consists of all terminal states. In that case, the simulation should stop, but this is not being checked at the moment and the action_info function leads to an error.

Please let me know if I am missing something here. Thanks!

zsunberg commented 1 month ago

The error here is due to an inaccurate belief. The person designing or operating the belief updater or policy should be responsible for handling this, not the simulation - it should be fixed at the solver/belief updater. If the simulator terminates on an inaccurate "terminal" belief, it could prevent the simulation from reaching an important terminal state, which would be a silent error and hence harmful.