search

JuliaPOMDP / DiscreteValueIteration.jl

Value iteration solver for MDPs
Other
20 stars 12 forks source link

How to trac iteration in the solver #13

Closed chetankul closed 7 years ago

chetankul commented 7 years ago

@zsunberg I am trying to track iteration steps in the solver. At a specific step number I want to switch to a different reward location GridState.

Can you please help on how this can be setup

Thanks

zsunberg commented 7 years ago

Hi @chetankul, I'm not 100% sure that I understand your question. Do you want to 1) change the MDP definition at some point in the solution process or 2) change the goal state after some number of steps in the MDP?

(1) is not supported in this solver. If you want to do that, I would recommend creating your own fork of the code. Also, if you are trying to do (1), why do you want to do that? I'm curious. If you want to do (2), you should augment the state to include the time step and then modify the reward function to give reward to one state if the time step is less than some value and another state if it is greater. Does that make sense?

zsunberg commented 7 years ago

Actually, you could do (1) by setting max_iterations in the solver to the number that you want to switch at, running the solver for that many steps, creating a new MDP with the new goal state, seeding the solver with the qmat from the old solution, and then proceeding from there with the solution. Does that make sense?