search

JuliaPOMDP / FIB.jl

Fast informed bound solver for POMDPs.
Other
2 stars 3 forks source link

Support state-dependent action space #6

Open zsunberg opened 5 years ago

zsunberg commented 5 years ago

Not sure if it is easy to support action spaces in this solver, but that would be a good modification for someone to do. It is not currently supported.

dressel commented 5 years ago

I'm not entirely sure how we would do this. For example, we take a maximum over a', which is the action you would take at s'. The set of available actions would be different for all your s' in your sum, and I'm not sure how to handle that theoretically. Also, how do you update alpha_a for an action s that doesn't support action a?

zsunberg commented 5 years ago

Oh, ok. I see. Yeah, it's not obvious how to do it.

zsunberg commented 5 years ago

Would it make more sense to have it be observation-dependent? That would have bearing on JuliaPOMDP/POMDPs.jl#226. It does not seem very obvious to me

dressel commented 5 years ago

The more I think about it, do state-dependent action spaces make sense for POMDPs? As you generally don't know your state so it's not like you can limit your actions. I think it makes more sense to just have actions not do anything (have no effect) from certain states.

It seems like observation-dependent action spaces might be feasible. In the summation over observations, you can just take the maximum over the a' that you can take having seen that observation.

I probably won't add this, but if anyone wants to do so (and to take a rigorous theoretical look to make sure it is indeed ok), I'd be ok with that.

zsunberg commented 5 years ago

Yeah - the question of whether state-dependent actions even make sense for POMDPs is key in JuliaPOMDP/POMDPs.jl#226. As you say, you don't know your state, so it makes more sense to make the invalid actions do nothing/ have negative consequences.