search

JuliaPOMDP / POMDPs.jl

MDPs and POMDPs in Julia - An interface for defining, solving, and simulating fully and partially observable Markov decision processes on discrete and continuous spaces.
http://juliapomdp.github.io/POMDPs.jl/latest/
Other
657 stars 100 forks source link

SparseTabularMDP doesn't support initialstate #444

Closed fredcallaway closed 1 year ago

fredcallaway commented 1 year ago

If I define an MDP with QuickMDP, I can specify an initialstate distribution which is automatically used in e.g. RolloutSimulator. But when I convert the QuickMDP to a SparseTabularMDP (for faster rollouts) the initialstate is lost. Any particular reason for this?

zsunberg commented 1 year ago

Upon looking, I see that SparseTabularMDP does not even have a field for representing the initial distribution.

zsunberg commented 1 year ago

Someone should make a pull request to add this. The probable reason that this was not added was that usually the sparse tabular representation is used for value iteration or similar offline methods that do not need an initial state distribution. But someone should add it for simulation purposes

fredcallaway commented 1 year ago

Cool, I'm on it