About MCTSnet

[tensorstart]

Hello, have you successfully replaced MCTS with MCTSnet?I have applied MCTSnet in the Sokoban, which is a GYM based environment, but the effect is not very good. The accuracy rate of each step is only 70%, and the victory rate is only 3%. Could you tell me how your effect is，thx.

[kblomdahl]

Hi, I have not had much success with MCTSnet. I have a half implementation [1] but I have encountered numerous problems that made me pause the implementation for the moment:

• Unsatisfying performance. This is partially an implementation detail, but due to the limited batch size I do not get good GPU utilization which limits its training.
• The accuracy of the MCTSnet policy did not converge to a better solution than a traditional policy network.
• The networks of the original article did not work, they did not converge at all for non-trivial data. I had to change all of them to GRU cells which is unsatisfying.

I have not taken the project further than the training step, so I do not know how it would affect the actual playing strength.

[1] https://github.com/Chicoryn/dream-go/tree/dev/mctsnet/contrib/mctsnet

[tensorstart]

Thank you for your reply, we have encountered similar problems with you. Our GPU utilization is also not high. Although Deepmind's ideas are interesting, they don't give any reference code. We were surprised to see that they used MCTSnet to perform a simulation training 3 million times in a Sokoban environment to achieve a 60% success rate.