WIP: Add `action(policy, s)` interface to exploration policies

JuliaPOMDP / POMDPs.jl

MDPs and POMDPs in Julia - An interface for defining, solving, and simulating fully and partially observable Markov decision processes on discrete and continuous spaces.

http://juliapomdp.github.io/POMDPs.jl/latest/

Other

664 stars 100 forks source link

WIP: Add `action(policy, s)` interface to exploration policies #510

Open johannes-fischer opened 1 year ago

johannes-fischer commented 1 year ago

To address https://github.com/JuliaPOMDP/POMDPs.jl/issues/497, I extended EpsGreedyPolicy by internal fields for the greedy policy and the iteration k. If this seems like a reasonable way to go, I can make similar changes to SoftmaxPolicy and edit doc strings. If not feel free to close this PR.

With this change both interfaces are supported: action(p::EpsGreedyPolicy, on_policy::Policy, k, s) action(p::EpsGreedyPolicy, s)

zsunberg commented 1 year ago

Thanks for the contribution!

My only concern is that it goes against the "preferably only one obvious way to do it" wisdom (https://peps.python.org/pep-0020/), since now k is in multiple places.

But I think this is probably OK. I think you should go ahead and implement it for the other policies and I will merge if it looks good.

zsunberg commented 1 year ago

Is there a name we could use besides update!? Perhaps something more specific like set_age!