search

yhyu13 / AlphaGOZero-python-tensorflow

Congratulation to DeepMind! This is a reengineering implementation (on behalf of many other git repo in /support/) of DeepMind's Oct19th publication: [Mastering the Game of Go without Human Knowledge]. The supervised learning approach is more practical for individuals. (This repository has single purpose of education only)
MIT License
341 stars 115 forks source link

Multi gpu #3

Closed yhyu13 closed 6 years ago

yhyu13 commented 6 years ago

This is a parallel multi-gpu implementaion inspired by cifar10_multi_gpu_train.py

  1. The idea is to split mini_batch (assume size 2048) input mini_mini_batch (assume 4 GPU, each GPU feeds batch size 512).
  2. Each GPU then compute loss & gradients and is updated by their average. Each GPU shares the same weights (reuse=True).