search

JuliaPOMDP / BasicPOMCP.jl

The PO-UCT algorithm (aka POMCP) implemented in Julia
Other
35 stars 17 forks source link

No rollout allowed from the root of a tree #20

Closed gautams3 closed 4 years ago

gautams3 commented 4 years ago

Ref: BasicPOMCP/solver.jl::simulate()

If you start at the root of a POMCP tree, you cannot directly jump to a rollout. You have to do selection based on upper-confidence bound (UCB) at least once before calling a rollout. The POMCP paper doesn't require that, as per Algorithm 1.

[1] Silver, D., & Veness, J. (2010). Monte-Carlo Planning in Large POMDPs. In Advances in neural information processing systems (pp. 2164–2172). Retrieved from http://discovery.ucl.ac.uk/1347369/

zsunberg commented 4 years ago

You are correct that there is a difference, however I would argue that it is inconsequential. In the algorithm presented in the paper, that first rollout is just thrown away, correct?

zsunberg commented 4 years ago

P.S. this code was written in the early days of julia and I think could be made much cleaner.

gautams3 commented 4 years ago

I looked at the algorithm again. You're right, the very first rollout is not used for any value update.