SuperGo

A student implementation of AlphaGo Zero paper with documentation.

Ongoing project.

TODO (in order of priority)

• Do something about the process leaking
• File of constants that match the paper constants ?
• OGS / KGS API ?
• Use logging instead of prints ?

CURRENTLY DOING

• Optimizations
• Clean code, create install script, write documentation
• Trying to see if it learns something on my computer

DONE

• Statistics (branch statistics)
• Game that are longer than the threshold of moves are now used
• MCTS
  • Tree search
  • Dirichlet noise to prior probabilities in the rootnode
  • Adaptative temperature (either take max or proportionally)
  • Sample random rotation or reflection in the dihedral group
  • Multithreading of search
  • Batch size evaluation to save computation
• Dihedral group of board for more training samples
• Learning without MCTS doesnt seem to work
• Resume training
• GTP on trained models (human.py, to plug with Sabaki)
• Learning rate annealing (see this)
• Better display for game (viewer.py, converting self-play games into GTP and then using Sabaki)
• Make the 3 components (self-play, training, evaluation) asynchronous
• Multiprocessing of games for self-play and evaluation
• Models and training without MCTS
• Evaluation
• Tromp Taylor scoring
• Dataset ring buffer of self-play games
• Loading saved models
• Database for self-play games

LONG TERM PLAN ?

• Compile my own version of Sabaki to watch games automatically while traning
• Resignation ?
• Training on a big computer / server once everything is ready ?

Resources

• The article for this code
• Official AlphaGo Zero paper
• Custom environment implementation using pachi_py following the implementation that was originally made on OpenAI Gym
• Using PyTorch for the neural networks
• Using Sabaki for the GUI
• General scheme, cool design
• Monte Carlo tree search explaination
• Nice tree search implementation

Statistics, check branch stats

For a 10 layers deep Resnet

9x9 board

soon

19x19 board

Differences with the official paper

• No resignation
• PyTorch instead of Tensorflow
• Python instead of (probably) C++ / C