suragnair / alpha-zero-general

A clean implementation based on AlphaZero for any game in any framework + tutorial + Othello/Gobang/TicTacToe/Connect4 and more
MIT License
3.9k stars 1.04k forks source link

visit count initialization for the leaf node #326

Open eijoac opened 2 days ago

eijoac commented 2 days ago

In the search() method of the MCTS class, when a leaf node is reached, after the expansion, action prior probability and leaf state value initialization from the NN, shouldn't we set the leaf node visit count (self.Ns[s]) to 1 instead of 0 (line 101 in the MCTS.py here). This visit count isn't incremented during the backpropagation later, so I think it should be set to 1 at initialization.