pytorch / ELF

ELF: a platform for game research with AlphaGoZero/AlphaZero reimplementation
Other
3.37k stars 566 forks source link

When will it start to train? #61

Closed infinitycr closed 6 years ago

infinitycr commented 6 years ago

After 29000 games, it doesn't start to train.

=== Record Stats (0) ==== B/W/A: 14683/14317/29000 (50.631%). B #Resign: 11904 (41.0483%), W #Resign: 11770 (40.5862%), #NoResign: 5326 (18.3655%) Dynamic resign threshold: 0.01 Move: [0, 100): 0, [100, 200): 0, [200, 300): 0, [300, up): 29000 === End Record Stats ====

The script of server is like:

save=./myserver game=elfgames.go.game model=df_kl model_file=elfgames.go.df_model3 \ stdbuf -o 0 -e 0 python -u ./train.py \ --mode train --batchsize 2048 \ --num_games 64 --keys_in_reply V \ --T 1 --use_data_parallel \ --num_minibatch 1000 --num_episode 1000000 \ --mcts_threads 8 --mcts_rollout_per_thread 100 \ --keep_prev_selfplay --keep_prev_selfplay \ --use_mcts --use_mcts_ai2 \ --mcts_persistent_tree --mcts_use_prior \ --mcts_virtual_loss 5 --mcts_epsilon 0.25 \ --mcts_alpha 0.03 --mcts_puct 0.85 \ --resign_thres 0.01 --gpu 0 \ --server_id myserver --eval_num_games 400 \ --eval_winrate_thres 0.55 --port 1234 \ --q_min_size 200 --q_max_size 4000 \ --save_first \ --num_block 5 --dim 64 \ --weight_decay 0.0002 --opt_method sgd \ --bn_momentum=0 --num_cooldown=50 \ --expected_num_client 496 \ --selfplay_init_num 0 --selfplay_update_num 0 \ --eval_num_games 0 --selfplay_async \ --lr 0.01 --momentum 0.9 1>> log.log 2>&1 &

I want to know when will it start to train?

infinitycr commented 6 years ago

The directory of ELF/scripts/elfgames/go is like:

analysis.sh train.py console_lib.py df_console.py
gtp.sh log.log myserver pycache selfplay.log selfplay.py server_addrs.py start_client.sh start_selfplay.sh start_server.sh selfplay-myserver-180605-105504-0-14-0.json selfplay-myserver-180605-105504-0-28-0.json
selfplay-myserver-180605-105504-0-15-0.json selfplay-myserver-180605-105504-0-3-0.json
selfplay-myserver-180605-105504-0-16-0.json selfplay-myserver-180605-105504-0-4-0.json
selfplay-myserver-180605-105504-0-17-0.json selfplay-myserver-180605-105504-0-5-0.json
selfplay-myserver-180605-105504-0-18-0.json selfplay-myserver-180605-105504-0-6-0.json
selfplay-myserver-180605-105504-0-19-0.json selfplay-myserver-180605-105504-0-7-0.json
selfplay-myserver-180605-105504-0-20-0.json selfplay-myserver-180605-105504-0-8-0.json
selfplay-myserver-180605-105504-0-2-0.json selfplay-myserver-180605-105504-0-9-0.json
selfplay-myserver-180605-105504-0-21-0.json selfplay-myserver-180605-105504-0-0-0.json
selfplay-myserver-180605-105504-0-22-0.json selfplay-myserver-180605-105504-0-10-0.json
selfplay-myserver-180605-105504-0-23-0.json selfplay-myserver-180605-105504-0-1-0.json
selfplay-myserver-180605-105504-0-24-0.json selfplay-myserver-180605-105504-0-11-0.json
selfplay-myserver-180605-105504-0-25-0.json selfplay-myserver-180605-105504-0-12-0.json
selfplay-myserver-180605-105504-0-26-0.json selfplay-myserver-180605-105504-0-13-0.json
selfplay-myserver-180605-105504-0-27-0.json

There is no directory record. Is this correct? Or it cannot start to train because of missing directory record?

qucheng commented 6 years ago

By default we use num_reader * q_max_size as the initial replay buffer size, default is 50x1000. Here you are using 50x4000 = 200k, so there needs to be 200k games for the first training iteration to start.