search

eleurent / rl-agents

Implementations of Reinforcement Learning and Planning algorithms
MIT License
582 stars 152 forks source link

mcts with dpw #45

Closed saArbabi closed 4 years ago

saArbabi commented 4 years ago

What?

I have added progressive widening (pw) for both actions and states.

Why?

Since action space is relatively small, it does not make much difference. However, state pw allows mcts to handle continuous transitions support. Without state pw, construction of the tree would not reach beyond depth 1.

How?

Inherited from tree search abstract classes. Added decision nodes and chance nodes. The added code for progressive widening is in the get_child() methods.

Testing?

Minor testing by adding gaussian noise to acceleration of IDM agents.

eleurent commented 4 years ago

Thanks a lot @saArbabi, that is a really nice addition! I have a minor comment about the implementation: to avoid duplicated code, you could make the MCTSDPW class inherit from MCTS rather than AbstractPlanner, and MCTSDPWAgent from MCTSAgent rather than AbstractTreeSearchAgent. And same for DecisionNodeand ChanceNode. That would lighten the file to only include the modification of node expansions with DPW.

saArbabi commented 4 years ago

Hi eleurent. Thanks for the review. I should have done that in the first place. Please let me know if it looks ok now.

eleurent commented 4 years ago

Thank you for this! It looks great, I'll try it soon.