I think we have a problem. I played this game:

https://lichess.org/nzvZOugM LC0 lost with blunders!! (Network Id. 43 vs. Arham555 - Chess.com ELO: 1461 - 20 sec/movement)

LC0 made a lot of blunders in this game, hang up pieces, etc.

I used "go movetime 20000", i.e. 20 seconds per game.

I run the model in Arena with: lczero.exe --weights 8a288da9757ed4361b84f8b3f74e07a37d3309893da4e8235811538ac59e1e45 -q -t1

Maybe it was a bug in the movetime command of the UCI inside our project? (maybe that truncate the search tree in a bad way or something like that).

it's strange, the model had never failed in this way before, I guess if it was due to the use of go movetime UCI command or if it's a fails of the model play.

I paste over here the thinking history:

2018-03-27 08:13:24.887*----------New game---2018-03-27 08:13:24,887 Tue ------------- 2018-03-27 08:13:41.9271Start calc, move no: 1 2018-03-27 08:13:41.928-->1:ucinewgame 2018-03-27 08:13:41.929-->1:isready 2018-03-27 08:13:41.930<--1:Unknown command: ucinewgame ucinewgame 2018-03-27 08:13:41.930<--1:readyok 2018-03-27 08:13:41.942-->1:position startpos moves e2e4 2018-03-27 08:13:41.943-->1:go movetime 20000 2018-03-27 08:14:01.897<--1:info depth 20 nodes 6115 nps 306 score cp -33 winrate 48.70% time 19953 pv e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 f8e7 f1e2 e8g8 2018-03-27 08:14:01.899<--1:bestmove e7e5 2018-03-27 08:14:01.9001Found move:e7-e5 2018-03-27 08:18:34.2571Start calc, move no: 3 2018-03-27 08:18:34.257-->1:position startpos moves e2e4 e7e5 g1f3 2018-03-27 08:18:34.257-->1:go movetime 20000 2018-03-27 08:18:54.210<--1:info depth 20 nodes 6777 nps 339 score cp -44 winrate 48.29% time 19953 pv d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 d6d5 e4e5 f6e4 2018-03-27 08:18:54.225<--1:bestmove d7d6 2018-03-27 08:18:54.2251Found move:d7-d6 2018-03-27 08:20:11.5551Start calc, move no: 5 2018-03-27 08:20:11.556-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 2018-03-27 08:20:11.556-->1:go movetime 20000 2018-03-27 08:20:31.508<--1:info depth 21 nodes 6709 nps 336 score cp -25 winrate 49.00% time 19951 pv e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 d4f5 c8f5 2018-03-27 08:20:31.510<--1:bestmove e5d4 2018-03-27 08:20:31.5111Found move:e5xd4 2018-03-27 08:21:40.6781Start calc, move no: 7 2018-03-27 08:21:40.679-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 2018-03-27 08:21:40.680-->1:go movetime 20000 2018-03-27 08:22:00.634<--1:info depth 21 nodes 10253 nps 513 score cp -32 winrate 48.73% time 19954 pv g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 d4f5 d6d5 f5e7 2018-03-27 08:22:00.636<--1:bestmove g8f6 2018-03-27 08:22:00.6371Found move:Ng8-f6 2018-03-27 08:23:24.3761Start calc, move no: 9 2018-03-27 08:23:24.376-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 2018-03-27 08:23:24.376-->1:go movetime 20000 2018-03-27 08:23:44.335<--1:info depth 21 nodes 7511 nps 376 score cp -63 winrate 47.58% time 19953 pv c7c6 f1e2 f8e7 e1g1 e8g8 d4f5 c8f5 e4f5 d6d5 c1f4 b8d7 2018-03-27 08:23:44.336<--1:bestmove c7c6 2018-03-27 08:23:44.3361Found move:c7-c6 2018-03-27 08:26:50.4781Start calc, move no: 11 2018-03-27 08:26:50.479-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 2018-03-27 08:26:50.480-->1:go movetime 20000 2018-03-27 08:27:10.434<--1:info depth 21 nodes 7181 nps 359 score cp -81 winrate 46.86% time 19953 pv f8e7 e1g1 e8g8 d4f5 c8f5 e4f5 d6d5 g2g4 b8d7 g4g5 f6e4 c3e4 2018-03-27 08:27:10.436<--1:bestmove f8e7 2018-03-27 08:27:10.4361Found move:Bf8-e7 2018-03-27 08:27:39.8011Start calc, move no: 13 2018-03-27 08:27:39.802-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 2018-03-27 08:27:39.803-->1:go movetime 20000 2018-03-27 08:27:59.758<--1:info depth 20 nodes 6023 nps 301 score cp -90 winrate 46.54% time 19954 pv e8g8 g2g4 d6d5 e4e5 f6e4 c3e4 d5e4 d4f5 d8d1 f1d1 e7c5 c1e3 c5e3 2018-03-27 08:27:59.760<--1:bestmove e8g8 2018-03-27 08:27:59.7601Found move:O-O 2018-03-27 08:28:12.5201Start calc, move no: 15 2018-03-27 08:28:12.521-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 2018-03-27 08:28:12.522-->1:go movetime 20000 2018-03-27 08:28:32.476<--1:info depth 20 nodes 5425 nps 271 score cp -25 winrate 49.00% time 19952 pv d6d5 e4e5 f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 b8d7 e5e6 d7e5 e6f7 2018-03-27 08:28:32.479<--1:bestmove d6d5 2018-03-27 08:28:32.4791Found move:d6-d5 2018-03-27 08:28:46.0431Start calc, move no: 17 2018-03-27 08:28:46.044-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 2018-03-27 08:28:46.044-->1:go movetime 20000 2018-03-27 08:29:05.996<--1:info depth 21 nodes 7519 nps 376 score cp -55 winrate 47.88% time 19951 pv a7a5 e5f6 e7f6 d4f3 b8a6 f3e5 a6c5 c1e3 c5e4 c3e4 2018-03-27 08:29:05.998<--1:bestmove a7a5 2018-03-27 08:29:05.9991Found move:a7-a5 2018-03-27 08:29:27.8981Start calc, move no: 19 2018-03-27 08:29:27.899-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 2018-03-27 08:29:27.899-->1:go movetime 20000 2018-03-27 08:29:47.853<--1:info depth 21 nodes 7150 nps 358 score cp -30 winrate 48.82% time 19952 pv e7f6 e2f3 g7g6 d4e2 f6g7 g2g4 b8d7 f3g2 d7f6 2018-03-27 08:29:47.855<--1:bestmove e7f6 2018-03-27 08:29:47.8551Found move:Be7xf6 2018-03-27 08:30:00.1951Start calc, move no: 21 2018-03-27 08:30:00.196-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 2018-03-27 08:30:00.196-->1:go movetime 20000 2018-03-27 08:30:20.150<--1:info depth 21 nodes 8452 nps 423 score cp -53 winrate 47.94% time 19952 pv g7g6 d4e2 f6g7 g2g4 b8d7 g4g5 d7b6 a2a4 2018-03-27 08:30:20.152<--1:bestmove g7g6 2018-03-27 08:30:20.1521Found move:g7-g6 2018-03-27 08:30:34.5151Start calc, move no: 23 2018-03-27 08:30:34.516-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 2018-03-27 08:30:34.517-->1:go movetime 20000 2018-03-27 08:30:54.469<--1:info depth 20 nodes 5526 nps 276 score cp -21 winrate 49.17% time 19952 pv f6g7 d4e2 b8a6 e2g3 a6c5 c1e3 c5e6 f4f5 2018-03-27 08:30:54.471<--1:bestmove f6g7 2018-03-27 08:30:54.4721Found move:Bf6-g7 2018-03-27 08:31:09.4141Start calc, move no: 25 2018-03-27 08:31:09.415-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 2018-03-27 08:31:09.416-->1:go movetime 20000 2018-03-27 08:31:29.369<--1:info depth 20 nodes 5395 nps 270 score cp 29 winrate 51.15% time 19952 pv b8d7 d4e2 d7f6 e2g3 h7h5 e3c5 f8e8 e1e8 d8e8 c5d4 h5h4 g3e2 2018-03-27 08:31:29.371<--1:bestmove b8d7 2018-03-27 08:31:29.3721Found move:Nb8-d7 2018-03-27 08:31:58.5521Start calc, move no: 27 2018-03-27 08:31:58.553-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 2018-03-27 08:31:58.554-->1:go movetime 20000 2018-03-27 08:32:18.509<--1:info depth 20 nodes 5635 nps 282 score cp 68 winrate 52.63% time 19954 pv d7f6 g2g4 h7h6 f4f5 f8e8 f5g6 f7g6 e3f4 e8e1 d1e1 2018-03-27 08:32:18.512<--1:bestmove d7f6 2018-03-27 08:32:18.5121Found move:Nd7-f6 2018-03-27 08:32:33.5901Start calc, move no: 29 2018-03-27 08:32:33.591-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 2018-03-27 08:32:33.592-->1:go movetime 20000 2018-03-27 08:32:53.545<--1:info depth 20 nodes 5917 nps 296 score cp 27 winrate 51.07% time 19952 pv c8f5 e3d4 f8e8 d4e5 h7h5 b3d4 f6g4 d4f5 g6f5 2018-03-27 08:32:53.547<--1:bestmove c8f5 2018-03-27 08:32:53.5481Found move:Bc8-f5 2018-03-27 08:33:10.8411Start calc, move no: 31 2018-03-27 08:33:10.842-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 2018-03-27 08:33:10.842-->1:go movetime 20000 2018-03-27 08:33:30.797<--1:info depth 20 nodes 5274 nps 264 score cp 87 winrate 53.34% time 19953 pv f5g4 f3g4 f6g4 e3d4 g4f6 h2h3 f8e8 g2g4 f6e4 c3e4 d5e4 d4g7 2018-03-27 08:33:30.799<--1:bestmove f5g4 2018-03-27 08:33:30.7991Found move:Bf5-g4 2018-03-27 08:33:57.1551Start calc, move no: 33 2018-03-27 08:33:57.156-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 2018-03-27 08:33:57.157-->1:go movetime 20000 2018-03-27 08:34:17.110<--1:info depth 21 nodes 6838 nps 342 score cp 58 winrate 52.23% time 19952 pv f6g4 e3d4 g4f6 h2h3 a8c8 g2g4 f8e8 e1e8 f6e8 d4g7 g8g7 2018-03-27 08:34:17.112<--1:bestmove f6g4 2018-03-27 08:34:17.1131Found move:Nf6xg4 2018-03-27 08:34:30.4961Start calc, move no: 35 2018-03-27 08:34:30.497-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 2018-03-27 08:34:30.498-->1:go movetime 20000 2018-03-27 08:34:50.451<--1:info depth 20 nodes 5674 nps 284 score cp 39 winrate 51.52% time 19952 pv f8e8 h2h3 g4f6 e1e8 f6e8 c5d4 g7d4 d2d4 e8f6 g2g4 h7h5 g4g5 2018-03-27 08:34:50.454<--1:bestmove f8e8 2018-03-27 08:34:50.4541Found move:Rf8-e8 2018-03-27 08:35:02.2531Start calc, move no: 37 2018-03-27 08:35:02.255-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 2018-03-27 08:35:02.256-->1:go movetime 20000 2018-03-27 08:35:22.210<--1:info depth 21 nodes 6841 nps 342 score cp 1 winrate 50.04% time 19953 pv d8e8 a1e1 e8d7 c5d4 a8f8 d4g7 g8g7 h2h3 g4f6 b3c5 d7c8 c5d3 f8e8 d3e5 c8f5 2018-03-27 08:35:22.212<--1:bestmove d8e8 2018-03-27 08:35:22.2131Found move:Qd8xe8 2018-03-27 08:35:34.4561Start calc, move no: 39 2018-03-27 08:35:34.457-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 2018-03-27 08:35:34.458-->1:go movetime 20000 2018-03-27 08:35:54.410<--1:info depth 21 nodes 10486 nps 525 score cp -53 winrate 47.95% time 19951 pv e8d7 h2h3 g4f6 c3a2 d7d8 d2e3 f6e4 c5d4 d8h4 d4g7 g8g7 2018-03-27 08:35:54.412<--1:bestmove e8d7 2018-03-27 08:35:54.4121Found move:Qe8-d7 2018-03-27 08:36:08.3341Start calc, move no: 41 2018-03-27 08:36:08.335-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 2018-03-27 08:36:08.336-->1:go movetime 20000 2018-03-27 08:36:28.291<--1:info depth 21 nodes 9415 nps 471 score cp -100 winrate 46.16% time 19954 pv g4f6 c3a2 a8e8 e1e8 f6e8 d2a5 e8d6 b3d4 d6e4 c5b6 g7d4 b6d4 c6c5 2018-03-27 08:36:28.293<--1:bestmove g4f6 2018-03-27 08:36:28.2931Found move:Ng4-f6 2018-03-27 08:36:45.8841Start calc, move no: 43 2018-03-27 08:36:45.885-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 2018-03-27 08:36:45.886-->1:go movetime 20000 2018-03-27 08:37:05.841<--1:info depth 20 nodes 5358 nps 268 score cp -58 winrate 47.75% time 19954 pv d7d8 d2e3 f6d7 g2g4 d7f8 e7b7 d8c8 b7e7 f8e6 f4f5 2018-03-27 08:37:05.843<--1:bestmove d7d8 2018-03-27 08:37:05.8431Found move:Qd7-d8 2018-03-27 08:37:18.5791Start calc, move no: 45 2018-03-27 08:37:18.580-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 2018-03-27 08:37:18.581-->1:go movetime 20000 2018-03-27 08:37:38.536<--1:info depth 21 nodes 7110 nps 356 score cp -84 winrate 46.76% time 19954 pv f6d7 c5d4 d8c8 b7a7 a8a7 d4a7 c8a6 a7d4 g7d4 d2d4 2018-03-27 08:37:38.538<--1:bestmove f6d7 2018-03-27 08:37:38.5381Found move:Nf6-d7 2018-03-27 08:37:53.3101Start calc, move no: 47 2018-03-27 08:37:53.312-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 2018-03-27 08:37:53.312-->1:go movetime 20000 2018-03-27 08:38:13.266<--1:info depth 20 nodes 5921 nps 296 score cp -188 winrate 42.79% time 19953 pv d8c8 b7a7 a8a7 d4a7 g7h6 g2g4 c8a8 a7d4 a8b8 g4g5 2018-03-27 08:38:13.269<--1:bestmove d8c8 2018-03-27 08:38:13.2691Found move:Qd8-c8 2018-03-27 08:38:35.7171Start calc, move no: 49 2018-03-27 08:38:35.719-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 d8c8 b7a7 2018-03-27 08:38:35.719-->1:go movetime 20000 2018-03-27 08:38:55.671<--1:info depth 21 nodes 8821 nps 442 score cp -197 winrate 42.47% time 19951 pv a8a7 d4a7 g7h6 g2g4 h6f8 b3a5 f8c5 a7c5 d7c5 2018-03-27 08:38:55.673<--1:bestmove a8a7 2018-03-27 08:38:55.6731Found move:Ra8xa7 2018-03-27 08:39:09.3691Start calc, move no: 51 2018-03-27 08:39:09.370-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 d8c8 b7a7 a8a7 d4a7 2018-03-27 08:39:09.370-->1:go movetime 20000 2018-03-27 08:39:29.324<--1:info depth 21 nodes 13012 nps 652 score cp -217 winrate 41.72% time 19952 pv g7h6 a7d4 d7f8 b3a5 f8e6 d4e5 h6f4 d2f4 e6f4 2018-03-27 08:39:29.326<--1:bestmove g7h6 2018-03-27 08:39:29.3271Found move:Bg7-h6 2018-03-27 08:39:47.2601Start calc, move no: 53 2018-03-27 08:39:47.261-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 d8c8 b7a7 a8a7 d4a7 g7h6 a7f2 2018-03-27 08:39:47.262-->1:go movetime 20000 2018-03-27 08:40:07.215<--1:info depth 20 nodes 5917 nps 296 score cp -310 winrate 38.28% time 19952 pv c8d8 g2g4 d7f8 b3d4 d8d7 g4g5 h6g7 d2d3 c6c5 2018-03-27 08:40:07.217<--1:bestmove c8d8 2018-03-27 08:40:07.2171Found move:Qc8-d8 2018-03-27 08:40:22.0811Start calc, move no: 55 2018-03-27 08:40:22.082-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 d8c8 b7a7 a8a7 d4a7 g7h6 a7f2 c8d8 c3e2 2018-03-27 08:40:22.082-->1:go movetime 20000 2018-03-27 08:40:42.036<--1:info depth 20 nodes 5820 nps 291 score cp -367 winrate 36.24% time 19953 pv d8c8 d2a5 d7f8 b3d4 h6g7 a5a7 c6c5 d4b5 2018-03-27 08:40:42.039<--1:bestmove d8c8 2018-03-27 08:40:42.0391Found move:Qd8-c8 2018-03-27 08:41:03.1401Start calc, move no: 57 2018-03-27 08:41:03.142-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 d8c8 b7a7 a8a7 d4a7 g7h6 a7f2 c8d8 c3e2 d8c8 b3a5 2018-03-27 08:41:03.142-->1:go movetime 20000 2018-03-27 08:41:23.097<--1:info depth 20 nodes 5921 nps 296 score cp -442 winrate 33.60% time 19954 pv d7f8 c2c4 d5c4 a5c4 c8e6 b2b3 h6g7 e2d4 e6d7 2018-03-27 08:41:23.099<--1:bestmove d7f8 2018-03-27 08:41:23.1001Found move:Nd7-f8 2018-03-27 08:41:39.5981Start calc, move no: 59 2018-03-27 08:41:39.599-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 d8c8 b7a7 a8a7 d4a7 g7h6 a7f2 c8d8 c3e2 d8c8 b3a5 d7f8 e2d4 2018-03-27 08:41:39.600-->1:go movetime 20000 2018-03-27 08:41:59.553<--1:info depth 20 nodes 5268 nps 264 score cp -527 winrate 30.74% time 19952 pv c6c5 d4c6 c8d7 f2c5 f8e6 b2b4 h6f4 d2d1 e6c5 b4c5 f4e3 g1h2 e3c5 2018-03-27 08:41:59.555<--1:bestmove c6c5 2018-03-27 08:41:59.5561Found move:c6-c5 2018-03-27 08:42:14.9851Start calc, move no: 61 2018-03-27 08:42:14.986-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 d8c8 b7a7 a8a7 d4a7 g7h6 a7f2 c8d8 c3e2 d8c8 b3a5 d7f8 e2d4 c6c5 d4c6 2018-03-27 08:42:14.987-->1:go movetime 20000 2018-03-27 08:42:34.942<--1:info depth 21 nodes 8830 nps 442 score cp -524 winrate 30.87% time 19954 pv c8d7 f2c5 f8e6 b2b4 h6f4 d2c3 f4h6 b4b5 d5d4 c5d4 e6d4 2018-03-27 08:42:34.944<--1:bestmove c8d7 2018-03-27 08:42:34.9451Found move:Qc8-d7 2018-03-27 08:43:01.6781Start calc, move no: 63 2018-03-27 08:43:01.679-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 d8c8 b7a7 a8a7 d4a7 g7h6 a7f2 c8d8 c3e2 d8c8 b3a5 d7f8 e2d4 c6c5 d4c6 c8d7 d2d5 2018-03-27 08:43:01.679-->1:go movetime 20000 2018-03-27 08:43:21.633<--1:info depth 20 nodes 5284 nps 264 score cp -705 winrate 25.25% time 19953 pv d7e8 d5c4 g6g5 f4f5 e8d7 g2g4 d7d2 a5b3 d2d1 2018-03-27 08:43:21.635<--1:bestmove d7e8 2018-03-27 08:43:21.6361Found move:Qd7-e8 2018-03-27 08:44:04.3861Start calc, move no: 65 2018-03-27 08:44:04.387-->1:position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 a7a5 e5f6 e7f6 e2f3 g7g6 f1e1 f6g7 c1e3 b8d7 a2a4 d7f6 d4b3 c8f5 d1d2 f5g4 f3g4 f6g4 e3c5 f8e8 e1e8 d8e8 a1e1 e8d7 h2h3 g4f6 e1e7 d7d8 e7b7 f6d7 c5d4 d8c8 b7a7 a8a7 d4a7 g7h6 a7f2 c8d8 c3e2 d8c8 b3a5 d7f8 e2d4 c6c5 d4c6 c8d7 d2d5 d7e8 f4f5 2018-03-27 08:44:04.388-->1:go movetime 20000 2018-03-27 08:44:24.341<--1:info depth 20 nodes 5155 nps 258 score cp -564 winrate 29.55% time 19952 pv g6f5 d5e5 f8e6 e5f5 h6g7 f5e4 e8d7 a5c4 d7c7 a4a5 2018-03-27 08:44:24.343<--1:bestmove g6f5 2018-03-27 08:44:24.3431*Found move:g6xf5

I just looked at your position with the first major blunder, 9 ...a5.

lczero -w ~/Downloads/LCZero-Networks/8a288d -p 800 -t1 --noponder
Using 1 thread(s).
Generated 1924 moves
Detecting residual layers...v1...64 channels...6 blocks.
position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5
go
RNG seed: 0xb7d0f9442339108d (thread: 10181866982453914292)
info depth 7 nodes 1 nps 58 score cp -61 winrate 47.64% time 16 pv f6e4 c3e4
info depth 8 nodes 3 nps 85 score cp -25 winrate 49.03% time 34 pv f6e4 c3e4 d5e4
info depth 9 nodes 5 nps 100 score cp 0 winrate 49.97% time 49 pv f6e4 c3e4 d5e4 f4f5
info depth 10 nodes 9 nps 108 score cp -12 winrate 49.51% time 82 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3
info depth 11 nodes 17 nps 116 score cp -78 winrate 46.98% time 145 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5
info depth 12 nodes 32 nps 128 score cp -82 winrate 46.84% time 249 pv f6e4 c3e4 d5e4 c1e3 g7g6 f4f5
info depth 13 nodes 58 nps 138 score cp -66 winrate 47.44% time 417 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b3 b8c6
info depth 14 nodes 104 nps 143 score cp -81 winrate 46.86% time 724 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8
info depth 15 nodes 188 nps 142 score cp -68 winrate 47.38% time 1315 pv f6e4 c3e4 d5e4 c1e3 e7c5 f4f5 d8h4
info depth 16 nodes 341 nps 148 score cp -72 winrate 47.22% time 2300 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8
info depth 17 nodes 626 nps 150 score cp -55 winrate 47.87% time 4158 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 f8d8

f6e4 ->     365 (V: 48.76%) (N: 37.81%) PV: f6e4 c3e4 d5e4 c1e3 h7h6 f4f5 c6c5 d4b5 b8c6
b8d7 ->     125 (V: 50.16%) (N:  5.49%) PV: b8d7 e5f6 d7f6 f4f5 e7c5 c3a4 c5d4
f6d7 ->     123 (V: 49.42%) (N:  9.38%) PV: f6d7 d4f5 e7c5 g1h1 g7g6 f5h6 g8g7 h6g4 d8h4
a7a5 ->      41 (V: 48.95%) (N:  3.37%) PV: a7a5 e5f6 e7f6 d4f3 f8e8 a2a4 h7h6
h7h6 ->      18 (V: 46.66%) (N:  3.61%) PV: h7h6 e5f6 e7f6 d4f3 a7a5 f3e5 b8d7
f6g4 ->      17 (V: 37.74%) (N: 10.03%) PV: f6g4 e2g4 c8g4 d1g4 g7g6 f4f5
g7g6 ->      15 (V: 47.49%) (N:  2.48%) PV: g7g6 e5f6 e7f6 d4f3 f6g7 f3e5 b8d7
a7a6 ->      15 (V: 46.10%) (N:  3.29%) PV: a7a6 e5f6 e7f6 d4f3 b7b5 f3e5
c6c5 ->      15 (V: 45.05%) (N:  3.83%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7
f6e8 ->      11 (V: 48.14%) (N:  1.43%) PV: f6e8 f4f5 b8d7 e5e6 d7e5
e7c5 ->      11 (V: 43.85%) (N:  3.18%) PV: e7c5 e5f6 d8f6 c1e3 f6e7 e3f2 b8d7
h7h5 ->       8 (V: 44.17%) (N:  2.65%) PV: h7h5 e5f6 e7f6 d4f3 f8e8 f3e5
b8a6 ->       7 (V: 45.04%) (N:  2.04%) PV: b8a6 e5f6 e7f6 d4f3 a6c5 f3e5
d8e8 ->       5 (V: 43.79%) (N:  1.61%) PV: d8e8 e5f6 e7f6 d4f3 b8a6 f3e5
f6h5 ->       4 (V: 39.96%) (N:  1.94%) PV: f6h5 e2h5 g7g6 h5f3 e7c5
f8e8 ->       2 (V: 39.08%) (N:  1.02%) PV: f8e8 e5f6 e7f6
d8b6 ->       2 (V: 37.16%) (N:  1.02%) PV: d8b6 e5f6 e7f6
g8h8 ->       1 (V: 40.49%) (N:  0.54%) PV: g8h8 e5f6
d8a5 ->       1 (V: 39.81%) (N:  0.21%) PV: d8a5 e5f6
b7b6 ->       1 (V: 39.16%) (N:  0.61%) PV: b7b6 e5f6
d8d7 ->       1 (V: 37.97%) (N:  0.63%) PV: d8d7 e5f6
d8c7 ->       1 (V: 37.96%) (N:  0.31%) PV: d8c7 e5f6
e7b4 ->       1 (V: 37.86%) (N:  0.48%) PV: e7b4 e5f6
c8d7 ->       1 (V: 37.75%) (N:  0.69%) PV: c8d7 e5f6
c8e6 ->       1 (V: 37.21%) (N:  0.18%) PV: c8e6 e5f6
b7b5 ->       1 (V: 35.87%) (N:  0.92%) PV: b7b5 e5f6
e7d6 ->       1 (V: 35.54%) (N:  0.10%) PV: e7d6 e5d6
c8g4 ->       1 (V: 30.58%) (N:  0.47%) PV: c8g4 e2g4
e7a3 ->       1 (V: 30.01%) (N:  0.10%) PV: e7a3 e5f6
c8f5 ->       1 (V: 29.99%) (N:  0.13%) PV: c8f5 d4f5
g7g5 ->       1 (V: 29.66%) (N:  0.22%) PV: g7g5 e5f6
c8h3 ->       1 (V: 25.40%) (N:  0.18%) PV: c8h3 e5f6
d8d6 ->       1 (V: 13.10%) (N:  0.05%) PV: d8d6 e5d6

info depth 17 nodes 800 nps 152 score cp -49 winrate 48.08% time 5234 pv f6e4 c3e4 d5e4 c1e3 h7h6 f4f5 c6c5 d4b5 b8c6
bestmove f6e4

As you can see, even at 800 playout Leela should not play this move, PV is 9 ...Ne4. I have no idea what caused it to blunder in the game like this.

Do you think it could be due to the use of the UCI command "go movetime xxxx"? This is the only change I did today before the model started doing these things. Well, I also updated the model to the 8a288d one.

Hmm OK that's very interesting: At 4000 playouts, 9...a5 gains considerably, and at 20000 playouts it actually get played. Leela must be seeing something in that move but I have no idea what she thinks it will accomplish.

Analysis, 4000 playouts

``` lczero -w ~/Downloads/LCZero-Networks/8a288d -p 4000 -t1 --noponder Using 1 thread(s). Generated 1924 moves Detecting residual layers...v1...64 channels...6 blocks. position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 go RNG seed: 0x6c87b27bcefd8b71 (thread: 10181866982453914292) info depth 7 nodes 1 nps 58 score cp -61 winrate 47.64% time 16 pv f6e4 c3e4 info depth 8 nodes 3 nps 96 score cp -25 winrate 49.03% time 30 pv f6e4 c3e4 d5e4 info depth 9 nodes 5 nps 96 score cp 0 winrate 49.97% time 51 pv f6e4 c3e4 d5e4 f4f5 info depth 10 nodes 9 nps 109 score cp -12 winrate 49.51% time 81 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 info depth 11 nodes 17 nps 125 score cp -78 winrate 46.98% time 135 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 info depth 12 nodes 32 nps 133 score cp -82 winrate 46.84% time 239 pv f6e4 c3e4 d5e4 c1e3 g7g6 f4f5 info depth 13 nodes 58 nps 141 score cp -66 winrate 47.44% time 408 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b3 b8c6 info depth 14 nodes 104 nps 145 score cp -81 winrate 46.86% time 712 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 info depth 15 nodes 188 nps 147 score cp -68 winrate 47.38% time 1277 pv f6e4 c3e4 d5e4 c1e3 e7c5 f4f5 d8h4 info depth 16 nodes 341 nps 151 score cp -72 winrate 47.22% time 2251 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 info depth 17 nodes 626 nps 152 score cp -55 winrate 47.87% time 4105 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 f8d8 info depth 18 nodes 1143 nps 149 score cp -39 winrate 48.49% time 7623 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 b8d7 e5e6 d7e5 info depth 19 nodes 2076 nps 156 score cp -35 winrate 48.63% time 13287 pv f6e4 c3e4 d5e4 f4f5 e7c5 c2c3 b8d7 e5e6 d7e5 info depth 20 nodes 3744 nps 160 score cp -47 winrate 48.16% time 23264 pv f6e4 c3e4 d5e4 g2g4 h7h6 d4f5 e7c5 g1g2 d8d1 e2d1 f6e4 -> 1070 (V: 47.57%) (N: 37.81%) PV: f6e4 c3e4 d5e4 g2g4 g7g6 c1e3 c6c5 d4b5 b8c6 b8d7 -> 517 (V: 48.89%) (N: 5.49%) PV: b8d7 e5f6 d7f6 g2g4 h7h6 f4f5 e7d6 c1f4 d6f4 a7a5 -> 516 (V: 49.10%) (N: 3.37%) PV: a7a5 e5f6 e7f6 d4f3 b8d7 g2g4 f8e8 g4g5 g7g6 -> 456 (V: 49.16%) (N: 2.48%) PV: g7g6 e5f6 e7f6 d4f3 f6g7 f3e5 b8d7 e5d7 c8d7 a7a6 -> 371 (V: 48.96%) (N: 3.29%) PV: a7a6 e5f6 e7f6 d4f3 b8d7 f4f5 f8e8 c1f4 h7h6 -> 288 (V: 48.74%) (N: 3.61%) PV: h7h6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 f6d7 -> 249 (V: 47.43%) (N: 9.38%) PV: f6d7 d4f5 e7c5 g1h1 g7g6 f5h6 g8g7 h6g4 d5d4 c3e4 d8b6 -> 243 (V: 49.21%) (N: 1.02%) PV: d8b6 e5f6 e7f6 c1e3 f8e8 e3f2 b8d7 c3a4 b6d8 h7h5 -> 51 (V: 46.63%) (N: 2.65%) PV: h7h5 e5f6 e7f6 d4f3 b8d7 a2a4 a7a5 f6e8 -> 42 (V: 47.57%) (N: 1.43%) PV: f6e8 f4f5 e7c5 c1e3 d8h4 g2g3 f6g4 -> 40 (V: 36.14%) (N: 10.03%) PV: f6g4 e2g4 c8g4 d1g4 g7g6 f4f5 b8d7 b8a6 -> 36 (V: 46.50%) (N: 2.04%) PV: b8a6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 c6c5 -> 31 (V: 42.74%) (N: 3.83%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7 f3e5 e7c5 -> 29 (V: 43.66%) (N: 3.18%) PV: e7c5 e5f6 d8f6 c1e3 f6e7 e3f2 b8d7 d8e8 -> 14 (V: 43.12%) (N: 1.61%) PV: d8e8 e5f6 e7f6 d4f3 h7h6 f3e5 b8d7 f6h5 -> 10 (V: 39.79%) (N: 1.94%) PV: f6h5 e2h5 g7g6 h5f3 e7c5 c1e3 d8h4 f8e8 -> 7 (V: 42.21%) (N: 1.02%) PV: f8e8 e5f6 e7f6 d4f3 h7h6 f3e5 b7b5 -> 6 (V: 42.21%) (N: 0.92%) PV: b7b5 e5f6 e7f6 f4f5 b5b4 g8h8 -> 4 (V: 43.04%) (N: 0.54%) PV: g8h8 e5f6 e7f6 d4f3 h8g8 c8d7 -> 4 (V: 41.62%) (N: 0.69%) PV: c8d7 e5f6 e7f6 d4f3 f8e8 e7b4 -> 2 (V: 37.54%) (N: 0.48%) PV: e7b4 e5f6 d8f6 b7b6 -> 2 (V: 36.98%) (N: 0.61%) PV: b7b6 e5f6 e7f6 d8d7 -> 2 (V: 36.50%) (N: 0.63%) PV: d8d7 e5f6 e7f6 d8a5 -> 1 (V: 39.81%) (N: 0.21%) PV: d8a5 e5f6 d8c7 -> 1 (V: 37.96%) (N: 0.31%) PV: d8c7 e5f6 c8e6 -> 1 (V: 37.21%) (N: 0.18%) PV: c8e6 e5f6 e7d6 -> 1 (V: 35.54%) (N: 0.10%) PV: e7d6 e5d6 c8g4 -> 1 (V: 30.58%) (N: 0.47%) PV: c8g4 e2g4 e7a3 -> 1 (V: 30.01%) (N: 0.10%) PV: e7a3 e5f6 c8f5 -> 1 (V: 29.99%) (N: 0.13%) PV: c8f5 d4f5 g7g5 -> 1 (V: 29.66%) (N: 0.22%) PV: g7g5 e5f6 c8h3 -> 1 (V: 25.40%) (N: 0.18%) PV: c8h3 e5f6 d8d6 -> 1 (V: 13.10%) (N: 0.05%) PV: d8d6 e5d6 info depth 20 nodes 4000 nps 160 score cp -49 winrate 48.10% time 24999 pv f6e4 c3e4 d5e4 g2g4 g7g6 c1e3 c6c5 d4b5 b8c6 bestmove f6e4 ```

Analysis, 20000 playouts

``` lczero -w ~/Downloads/LCZero-Networks/8a288d -p 20000 -t1 --noponder Using 1 thread(s). Generated 1924 moves Detecting residual layers...v1...64 channels...6 blocks. position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 go RNG seed: 0xd3a26c417c301a0b (thread: 10181866982453914292) info depth 7 nodes 1 nps 58 score cp -61 winrate 47.64% time 16 pv f6e4 c3e4 info depth 8 nodes 3 nps 93 score cp -25 winrate 49.03% time 31 pv f6e4 c3e4 d5e4 info depth 9 nodes 5 nps 94 score cp 0 winrate 49.97% time 52 pv f6e4 c3e4 d5e4 f4f5 info depth 10 nodes 9 nps 116 score cp -12 winrate 49.51% time 76 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 info depth 11 nodes 17 nps 118 score cp -78 winrate 46.98% time 142 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 info depth 12 nodes 32 nps 130 score cp -82 winrate 46.84% time 244 pv f6e4 c3e4 d5e4 c1e3 g7g6 f4f5 info depth 13 nodes 58 nps 131 score cp -66 winrate 47.44% time 441 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b3 b8c6 info depth 14 nodes 104 nps 136 score cp -81 winrate 46.86% time 761 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 info depth 15 nodes 188 nps 145 score cp -68 winrate 47.38% time 1287 pv f6e4 c3e4 d5e4 c1e3 e7c5 f4f5 d8h4 info depth 16 nodes 341 nps 149 score cp -72 winrate 47.22% time 2282 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 info depth 17 nodes 626 nps 152 score cp -55 winrate 47.87% time 4091 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 f8d8 info depth 18 nodes 1143 nps 152 score cp -39 winrate 48.49% time 7489 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 b8d7 e5e6 d7e5 info depth 19 nodes 2076 nps 152 score cp -35 winrate 48.63% time 13615 pv f6e4 c3e4 d5e4 f4f5 e7c5 c2c3 b8d7 e5e6 d7e5 info depth 20 nodes 3744 nps 151 score cp -47 winrate 48.16% time 24681 pv f6e4 c3e4 d5e4 g2g4 h7h6 d4f5 e7c5 g1g2 d8d1 e2d1 info depth 21 nodes 6694 nps 155 score cp -59 winrate 47.71% time 43129 pv f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f5d6 c6d4 e2d1 info depth 22 nodes 12090 nps 163 score cp -44 winrate 48.28% time 74125 pv a7a5 e5f6 e7f6 d4f3 b8a6 f3e5 a6c5 c1e3 c5e6 f4f5 a7a5 -> 12667 (V: 48.84%) (N: 3.37%) PV: a7a5 e5f6 e7f6 e2f3 b8d7 g2g4 h7h6 d4f5 d7c5 c1e3 c5e6 g4g5 f6e4 -> 2489 (V: 47.04%) (N: 37.81%) PV: f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f5d6 c6d4 e2d1 d8h4 g7g6 -> 985 (V: 48.54%) (N: 2.48%) PV: g7g6 e5f6 e7f6 d4f3 f6g7 f3e5 b8d7 e5d7 c8d7 e2f3 d5d4 b8d7 -> 787 (V: 48.01%) (N: 5.49%) PV: b8d7 e5f6 d7f6 g2g4 h7h6 f4f5 e7d6 c1f4 d6f4 f6d7 -> 689 (V: 47.22%) (N: 9.38%) PV: f6d7 d4f5 e7c5 g1h1 a7a5 c3a4 b7b5 a4c5 d7c5 f5d6 a7a6 -> 501 (V: 48.05%) (N: 3.29%) PV: a7a6 e5f6 e7f6 d4f3 b8d7 g2g4 f8e8 g4g5 h7h6 -> 448 (V: 47.88%) (N: 3.61%) PV: h7h6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 d8b6 -> 270 (V: 48.37%) (N: 1.02%) PV: d8b6 e5f6 e7f6 c1e3 f8e8 e3f2 b8d7 c3a4 b6d8 h7h5 -> 260 (V: 47.64%) (N: 2.65%) PV: h7h5 e5f6 e7f6 d4f3 a7a5 a2a4 b8a6 f3e5 b8a6 -> 255 (V: 47.90%) (N: 2.04%) PV: b8a6 e5f6 e7f6 d4f3 a6c5 c1e3 c5e6 f4f5 d5d4 f8e8 -> 159 (V: 48.02%) (N: 1.02%) PV: f8e8 e5f6 e7f6 d4f3 h7h6 f3e5 f6e5 f4e5 f6g4 -> 101 (V: 36.88%) (N: 10.03%) PV: f6g4 e2g4 c8g4 d1g4 g7g6 f4f5 b8d7 e5e6 d7e5 f6e8 -> 96 (V: 47.05%) (N: 1.43%) PV: f6e8 f4f5 b8d7 e5e6 d7e5 c1f4 e7c5 -> 78 (V: 43.98%) (N: 3.18%) PV: e7c5 e5f6 d8f6 c1e3 f6e7 e3f2 b8d7 d4f5 e7f6 c6c5 -> 59 (V: 41.13%) (N: 3.83%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7 f3e5 b8c6 d8e8 -> 45 (V: 44.59%) (N: 1.61%) PV: d8e8 e5f6 e7f6 d4f3 b8d7 g2g4 h7h6 b7b5 -> 26 (V: 44.72%) (N: 0.92%) PV: b7b5 e5f6 e7f6 d4f3 b5b4 c3a4 f8e8 f6h5 -> 26 (V: 40.09%) (N: 1.94%) PV: f6h5 e2h5 g7g6 h5f3 e7c5 c1e3 d8h4 b7b6 -> 13 (V: 43.32%) (N: 0.61%) PV: b7b6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 c8d7 -> 12 (V: 42.34%) (N: 0.69%) PV: c8d7 e5f6 e7f6 d4f3 b8a6 f3e5 f6e5 g8h8 -> 9 (V: 42.18%) (N: 0.54%) PV: g8h8 e5f6 e7f6 d4f3 h8g8 f3e5 d8d7 -> 7 (V: 38.87%) (N: 0.63%) PV: d8d7 e5f6 e7f6 d4f3 d7d8 f3e5 e7b4 -> 5 (V: 37.93%) (N: 0.48%) PV: e7b4 e5f6 d8f6 d4f3 b4c3 d8a5 -> 2 (V: 37.23%) (N: 0.21%) PV: d8a5 e5f6 e7f6 d8c7 -> 2 (V: 33.88%) (N: 0.31%) PV: d8c7 e5f6 e7f6 c8g4 -> 2 (V: 28.93%) (N: 0.47%) PV: c8g4 e2g4 f6g4 c8e6 -> 1 (V: 37.21%) (N: 0.18%) PV: c8e6 e5f6 e7d6 -> 1 (V: 35.54%) (N: 0.10%) PV: e7d6 e5d6 e7a3 -> 1 (V: 30.01%) (N: 0.10%) PV: e7a3 e5f6 c8f5 -> 1 (V: 29.99%) (N: 0.13%) PV: c8f5 d4f5 g7g5 -> 1 (V: 29.66%) (N: 0.22%) PV: g7g5 e5f6 c8h3 -> 1 (V: 25.40%) (N: 0.18%) PV: c8h3 e5f6 d8d6 -> 1 (V: 13.10%) (N: 0.05%) PV: d8d6 e5d6 info depth 22 nodes 20000 nps 166 score cp -45 winrate 48.26% time 120271 pv a7a5 e5f6 e7f6 e2f3 b8d7 g2g4 h7h6 d4f5 d7c5 c1e3 c5e6 g4g5 bestmove a7a5 ```

It's very strange. Maybe it's just a failure in the model evaluation and nothing else. Could this be a regresion of the quality of the game strength?? Maybe we are overvaluating the real ELO gained by the model?

I think Leela had a plan with ...a5, but it just didn't work out. I think she must have thought it would lead to a good result. I wouldn't worry too much, future network generations will see that error and play differently again. For completeness' sake, here's the analysis with FPU reduction:

FPU reduction 800 playouts

``` lczero_fpu_puct08 -w ~/Downloads/LCZero-Networks/8a288d -p 800 -t1 --noponder Using 1 thread(s). Generated 1924 moves Detecting residual layers...v1...64 channels...6 blocks. position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 go RNG seed: 0xaf45c549ae8d5959 (thread: 10181866982453914292) info depth 7 nodes 1 nps 71 score cp -61 winrate 47.64% time 13 pv f6e4 c3e4 info depth 8 nodes 3 nps 111 score cp -30 winrate 48.81% time 26 pv f6e4 c3e4 d5e4 f4f5 info depth 9 nodes 5 nps 138 score cp -15 winrate 49.40% time 35 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 info depth 10 nodes 10 nps 128 score cp -135 winrate 44.81% time 77 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 d8h4 g2g3 h4h3 e2g4 h3g2 info depth 11 nodes 18 nps 135 score cp -102 winrate 46.05% time 132 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 d8h4 g2g3 h4h3 e2g4 h3g2 info depth 12 nodes 33 nps 147 score cp -77 winrate 47.01% time 223 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 d8h4 g2g3 h4h3 e2g4 h3g2 info depth 13 nodes 59 nps 153 score cp -90 winrate 46.52% time 383 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 f8d8 b5c7 a8b8 a1d1 info depth 14 nodes 105 nps 158 score cp -88 winrate 46.62% time 660 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5d6 e7d6 e5d6 b8c6 e3c5 info depth 15 nodes 190 nps 160 score cp -91 winrate 46.47% time 1185 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5d6 e7d6 e5d6 b8c6 e3c5 b7b6 info depth 16 nodes 345 nps 157 score cp -95 winrate 46.35% time 2185 pv f6e4 c3e4 d5e4 f4f5 c6c5 f5f6 c5d4 f6e7 d8e7 d1d4 b8c6 d4e4 info depth 17 nodes 630 nps 157 score cp -87 winrate 46.66% time 4002 pv f6e4 c3e4 d5e4 f4f5 g7g6 f5f6 e7c5 c1h6 d8d4 d1d4 c5d4 g1h1 d4e5 h6f8 g8f8 a1e1 f6e4 -> 440 (V: 47.18%) (N: 37.81%) PV: f6e4 c3e4 d5e4 f4f5 g7g6 f5f6 e7c5 c1h6 d8d4 d1d4 c5d4 g1h1 d4e5 h6f8 g8f8 a1e1 b8d7 -> 138 (V: 48.27%) (N: 5.49%) PV: b8d7 e5f6 d7f6 g2g4 h7h6 g4g5 h6g5 f4g5 f6e4 c3e4 d5e4 c2c3 e7g5 f6d7 -> 63 (V: 45.77%) (N: 9.38%) PV: f6d7 d4f5 e7c5 g1h1 d7b6 f5d6 c5d6 e5d6 d8d6 a2a4 b8d7 a4a5 b6c4 e2c4 d5c4 d1d6 d7f6 a7a5 -> 26 (V: 45.91%) (N: 3.37%) PV: a7a5 e5f6 e7f6 d4f3 h7h6 f3e5 b8d7 e5d7 c8d7 e2f3 d5d4 c3e4 f6e7 g7g6 -> 23 (V: 46.58%) (N: 2.48%) PV: g7g6 e5f6 e7f6 d4f3 f6g7 f3g5 h7h6 g5f3 f8e8 f3e5 b8d7 e5d7 c8d7 e2f3 h7h5 -> 17 (V: 45.12%) (N: 2.65%) PV: h7h5 e5f6 e7f6 e2h5 g7g6 h5f3 f6d4 d1d4 b8a6 d4e5 f7f6 e5e2 e7c5 -> 16 (V: 44.86%) (N: 3.18%) PV: e7c5 e5f6 d8f6 c1e3 b8d7 e2f3 f8e8 e3f2 f6f4 c3e2 f4e5 e2g3 c6c5 -> 15 (V: 43.20%) (N: 3.83%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7 f3e5 b8c6 e5c6 f6g4 -> 15 (V: 34.61%) (N: 10.03%) PV: f6g4 e2g4 c8g4 d1g4 g7g6 f4f5 b8d7 e5e6 d7e5 g4g3 e7h4 a7a6 -> 12 (V: 43.19%) (N: 3.29%) PV: a7a6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 c1f4 e5f5 e2g4 h7h6 -> 12 (V: 42.46%) (N: 3.61%) PV: h7h6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 c1f4 e5f5 e2g4 f5f6 b8a6 -> 9 (V: 44.26%) (N: 2.04%) PV: b8a6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 c1f4 d8e8 -> 6 (V: 43.26%) (N: 1.61%) PV: d8e8 e5f6 e7f6 d4f3 b8a6 f3e5 f6e5 f6e8 -> 4 (V: 42.55%) (N: 1.43%) PV: f6e8 f4f5 f7f6 e5e6 e7c5 f6h5 -> 4 (V: 39.96%) (N: 1.94%) PV: f6h5 e2h5 g7g6 h5f3 e7c5 d8b6 -> 0 (V: 51.27%) (N: 1.02%) PV: d8b6 f8e8 -> 0 (V: 51.27%) (N: 1.02%) PV: f8e8 b7b5 -> 0 (V: 51.27%) (N: 0.92%) PV: b7b5 c8d7 -> 0 (V: 51.27%) (N: 0.69%) PV: c8d7 d8d7 -> 0 (V: 51.27%) (N: 0.63%) PV: d8d7 b7b6 -> 0 (V: 51.27%) (N: 0.61%) PV: b7b6 g8h8 -> 0 (V: 51.27%) (N: 0.54%) PV: g8h8 e7b4 -> 0 (V: 51.27%) (N: 0.48%) PV: e7b4 c8g4 -> 0 (V: 51.27%) (N: 0.47%) PV: c8g4 d8c7 -> 0 (V: 51.27%) (N: 0.31%) PV: d8c7 g7g5 -> 0 (V: 51.27%) (N: 0.22%) PV: g7g5 d8a5 -> 0 (V: 51.27%) (N: 0.21%) PV: d8a5 c8e6 -> 0 (V: 51.27%) (N: 0.18%) PV: c8e6 c8h3 -> 0 (V: 51.27%) (N: 0.18%) PV: c8h3 c8f5 -> 0 (V: 51.27%) (N: 0.13%) PV: c8f5 e7d6 -> 0 (V: 51.27%) (N: 0.10%) PV: e7d6 e7a3 -> 0 (V: 51.27%) (N: 0.10%) PV: e7a3 d8d6 -> 0 (V: 51.27%) (N: 0.05%) PV: d8d6 info depth 17 nodes 800 nps 157 score cp -89 winrate 46.54% time 5067 pv f6e4 c3e4 d5e4 f4f5 g7g6 f5f6 e7c5 c1h6 d8d4 d1d4 c5d4 g1h1 d4e5 h6f8 g8f8 a1e1 bestmove f6e4 ```

FPU reduction 4000 playouts

``` lczero_fpu_puct08 -w ~/Downloads/LCZero-Networks/8a288d -p 4000 -t1 --noponder Using 1 thread(s). Generated 1924 moves Detecting residual layers...v1...64 channels...6 blocks. position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 go RNG seed: 0xcdb8506fe3031da8 (thread: 10181866982453914292) info depth 7 nodes 1 nps 71 score cp -61 winrate 47.64% time 13 pv f6e4 c3e4 info depth 8 nodes 3 nps 115 score cp -30 winrate 48.81% time 25 pv f6e4 c3e4 d5e4 f4f5 info depth 9 nodes 5 nps 128 score cp -15 winrate 49.40% time 38 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 info depth 10 nodes 10 nps 133 score cp -135 winrate 44.81% time 74 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 d8h4 g2g3 h4h3 e2g4 h3g2 info depth 11 nodes 18 nps 128 score cp -102 winrate 46.05% time 139 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 d8h4 g2g3 h4h3 e2g4 h3g2 info depth 12 nodes 33 nps 135 score cp -77 winrate 47.01% time 242 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 d8h4 g2g3 h4h3 e2g4 h3g2 info depth 13 nodes 59 nps 146 score cp -90 winrate 46.52% time 403 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 f8d8 b5c7 a8b8 a1d1 info depth 14 nodes 105 nps 150 score cp -88 winrate 46.62% time 696 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5d6 e7d6 e5d6 b8c6 e3c5 info depth 15 nodes 190 nps 155 score cp -91 winrate 46.47% time 1222 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5d6 e7d6 e5d6 b8c6 e3c5 b7b6 info depth 16 nodes 345 nps 155 score cp -95 winrate 46.35% time 2212 pv f6e4 c3e4 d5e4 f4f5 c6c5 f5f6 c5d4 f6e7 d8e7 d1d4 b8c6 d4e4 info depth 17 nodes 630 nps 155 score cp -87 winrate 46.66% time 4048 pv f6e4 c3e4 d5e4 f4f5 g7g6 f5f6 e7c5 c1h6 d8d4 d1d4 c5d4 g1h1 d4e5 h6f8 g8f8 a1e1 info depth 18 nodes 1159 nps 157 score cp -91 winrate 46.47% time 7358 pv f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f5d6 c6d4 e2d1 d8h4 c1e3 h7h5 info depth 19 nodes 2079 nps 159 score cp -94 winrate 46.39% time 13038 pv f6e4 c3e4 d5e4 f4f5 g7g6 f5f6 e7c5 c1h6 d8d4 d1d4 c5d4 g1h1 d4e5 h6f8 g8f8 a1e1 b8d7 info depth 20 nodes 3757 nps 164 score cp -92 winrate 46.43% time 22786 pv b8d7 e5f6 d7f6 g2g4 h7h6 g4g5 h6g5 f4g5 f6e4 c3e4 d5e4 c2c3 e7g5 c1g5 d8g5 g1h1 c8h3 f1g1 g5f6 b8d7 -> 1334 (V: 47.39%) (N: 5.49%) PV: b8d7 e5f6 d7f6 g2g4 h7h6 g4g5 h6g5 f4g5 f6e4 c3e4 d5e4 c2c3 e7g5 c1g5 d8g5 g1h1 c8h3 f1g1 g5f6 f6e4 -> 1160 (V: 45.96%) (N: 37.81%) PV: f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f5d6 c6d4 e2d1 d8h4 c1e3 h7h5 g4h5 c8h3 e3d4 c5d4 a7a5 -> 593 (V: 47.32%) (N: 3.37%) PV: a7a5 e5f6 e7f6 d4f3 b8d7 g2g4 h7h6 g4g5 h6g5 f4g5 f6e7 h2h4 e7c5 g1g2 d7b6 f3e5 f6d7 -> 317 (V: 46.10%) (N: 9.38%) PV: f6d7 d4f5 e7c5 g1h1 a7a5 c3a4 b7b5 a4c5 d7c5 f5d6 c8a6 f4f5 b8d7 c1f4 d8b6 e5e6 f7e6 f5e6 c5e6 d8b6 -> 118 (V: 47.59%) (N: 1.02%) PV: d8b6 e5f6 e7f6 c1e3 b6b2 c3a4 b2a3 e2f3 a3a4 d4b3 f6a1 d1a1 a7a6 -> 105 (V: 45.90%) (N: 3.29%) PV: a7a6 e5f6 e7f6 d4f3 b8d7 f3g5 f6g5 f4g5 d7e5 c1f4 e5g6 f4g3 h7h6 -> 76 (V: 45.09%) (N: 3.61%) PV: h7h6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 c1f4 e5f5 e2g4 f5f6 g4c8 e7c5 -> 52 (V: 44.37%) (N: 3.18%) PV: e7c5 e5f6 d8f6 c1e3 b8d7 e2f3 f8e8 e3f2 f6f4 c3e2 f4e5 e2g3 d7f6 f1e1 e5d6 f6g4 -> 43 (V: 36.11%) (N: 10.03%) PV: f6g4 e2g4 h7h6 g4c8 d8c8 f4f5 b8d7 e5e6 d7e5 c1f4 f7e6 c6c5 -> 32 (V: 41.56%) (N: 3.83%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7 f3e5 b8c6 e5c6 b7c6 f4f5 h7h5 -> 31 (V: 43.21%) (N: 2.65%) PV: h7h5 e5f6 e7f6 e2h5 g7g6 h5f3 f6d4 d1d4 b8a6 d4e5 f7f6 e5e2 f8e8 e2d1 c8f5 f6e8 -> 29 (V: 45.20%) (N: 1.43%) PV: f6e8 f4f5 f7f6 e5e6 e7c5 c1e3 d8b6 c3a4 b6a5 a4c5 a5c5 e2h5 b8a6 -> 28 (V: 44.06%) (N: 2.04%) PV: b8a6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 c1f4 e5f5 e2g4 f5f6 g7g6 -> 27 (V: 43.11%) (N: 2.48%) PV: g7g6 e5f6 e7f6 d4f3 f6g7 f3g5 h7h6 g5f3 f8e8 f3e5 b8d7 e5d7 c8d7 e2f3 f8e8 -> 11 (V: 42.37%) (N: 1.02%) PV: f8e8 e5f6 e7f6 d4f3 h7h6 f3e5 f6e5 f4e5 e8e5 c1f4 e5f5 e2g4 d8e8 -> 11 (V: 39.59%) (N: 1.61%) PV: d8e8 e5f6 e7f6 d4f3 b8a6 f3e5 f6e5 f4e5 e8e5 c1f4 e5f6 f4g3 b7b5 -> 9 (V: 42.28%) (N: 0.92%) PV: b7b5 e5f6 e7f6 d4f3 b5b4 c3a4 f8e8 a4c5 a7a5 f3e5 f6h5 -> 9 (V: 36.87%) (N: 1.94%) PV: f6h5 e2h5 g7g6 h5f3 e7c5 c1e3 d8h4 g2g3 h4h3 f3g2 g8h8 -> 6 (V: 43.62%) (N: 0.54%) PV: g8h8 e5f6 e7f6 d4f3 h8g8 f3e5 b8d7 c8d7 -> 5 (V: 41.15%) (N: 0.69%) PV: c8d7 e5f6 e7f6 d4f3 f8e8 f3e5 b7b6 -> 2 (V: 36.98%) (N: 0.61%) PV: b7b6 e5f6 e7f6 d8d7 -> 2 (V: 36.50%) (N: 0.63%) PV: d8d7 e5f6 e7f6 e7b4 -> 0 (V: 51.27%) (N: 0.48%) PV: e7b4 c8g4 -> 0 (V: 51.27%) (N: 0.47%) PV: c8g4 d8c7 -> 0 (V: 51.27%) (N: 0.31%) PV: d8c7 g7g5 -> 0 (V: 51.27%) (N: 0.22%) PV: g7g5 d8a5 -> 0 (V: 51.27%) (N: 0.21%) PV: d8a5 c8e6 -> 0 (V: 51.27%) (N: 0.18%) PV: c8e6 c8h3 -> 0 (V: 51.27%) (N: 0.18%) PV: c8h3 c8f5 -> 0 (V: 51.27%) (N: 0.13%) PV: c8f5 e7d6 -> 0 (V: 51.27%) (N: 0.10%) PV: e7d6 e7a3 -> 0 (V: 51.27%) (N: 0.10%) PV: e7a3 d8d6 -> 0 (V: 51.27%) (N: 0.05%) PV: d8d6 info depth 20 nodes 4000 nps 165 score cp -94 winrate 46.39% time 24124 pv b8d7 e5f6 d7f6 g2g4 h7h6 g4g5 h6g5 f4g5 f6e4 c3e4 d5e4 c2c3 e7g5 c1g5 d8g5 g1h1 c8h3 f1g1 g5f6 bestmove b8d7 ```

FPU reduction 20000 playouts

``` lczero_fpu_puct08 -w ~/Downloads/LCZero-Networks/8a288d -p 20000 -t1 --noponder Using 1 thread(s). Generated 1924 moves Detecting residual layers...v1...64 channels...6 blocks. position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 go RNG seed: 0x27f53aa4b7844375 (thread: 10181866982453914292) info depth 7 nodes 1 nps 52 score cp -61 winrate 47.64% time 18 pv f6e4 c3e4 info depth 8 nodes 3 nps 93 score cp -30 winrate 48.81% time 31 pv f6e4 c3e4 d5e4 f4f5 info depth 9 nodes 5 nps 98 score cp -15 winrate 49.40% time 50 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 info depth 10 nodes 10 nps 111 score cp -135 winrate 44.81% time 89 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 d8h4 g2g3 h4h3 e2g4 h3g2 info depth 11 nodes 18 nps 120 score cp -102 winrate 46.05% time 149 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 d8h4 g2g3 h4h3 e2g4 h3g2 info depth 12 nodes 33 nps 128 score cp -77 winrate 47.01% time 255 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 d8h4 g2g3 h4h3 e2g4 h3g2 info depth 13 nodes 59 nps 140 score cp -90 winrate 46.52% time 419 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 f8d8 b5c7 a8b8 a1d1 info depth 14 nodes 105 nps 147 score cp -88 winrate 46.62% time 709 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5d6 e7d6 e5d6 b8c6 e3c5 info depth 15 nodes 190 nps 144 score cp -91 winrate 46.47% time 1312 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5d6 e7d6 e5d6 b8c6 e3c5 b7b6 info depth 16 nodes 345 nps 146 score cp -95 winrate 46.35% time 2346 pv f6e4 c3e4 d5e4 f4f5 c6c5 f5f6 c5d4 f6e7 d8e7 d1d4 b8c6 d4e4 info depth 17 nodes 630 nps 142 score cp -87 winrate 46.66% time 4410 pv f6e4 c3e4 d5e4 f4f5 g7g6 f5f6 e7c5 c1h6 d8d4 d1d4 c5d4 g1h1 d4e5 h6f8 g8f8 a1e1 info depth 18 nodes 1159 nps 142 score cp -91 winrate 46.47% time 8119 pv f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f5d6 c6d4 e2d1 d8h4 c1e3 h7h5 info depth 19 nodes 2079 nps 144 score cp -94 winrate 46.39% time 14377 pv f6e4 c3e4 d5e4 f4f5 g7g6 f5f6 e7c5 c1h6 d8d4 d1d4 c5d4 g1h1 d4e5 h6f8 g8f8 a1e1 b8d7 info depth 20 nodes 3757 nps 152 score cp -92 winrate 46.43% time 24611 pv b8d7 e5f6 d7f6 g2g4 h7h6 g4g5 h6g5 f4g5 f6e4 c3e4 d5e4 c2c3 e7g5 c1g5 d8g5 g1h1 c8h3 f1g1 g5f6 info depth 21 nodes 6782 nps 157 score cp -96 winrate 46.31% time 43032 pv b8d7 e5f6 d7f6 g2g4 h7h6 g4g5 h6g5 f4g5 f6e4 c3e4 d5e4 c1f4 e7g5 f4g5 d8g5 g1h1 c8h3 f1g1 g5f6 g1g3 info depth 22 nodes 12080 nps 165 score cp -79 winrate 46.95% time 72785 pv a7a5 e5f6 e7f6 d4f3 b8d7 g2g4 d7b6 g4g5 f6e7 f3e5 f7f6 e5f3 f6g5 f4g5 c8h3 f1f2 e7c5 c1e3 a7a5 -> 12671 (V: 48.20%) (N: 3.37%) PV: a7a5 e5f6 e7f6 e2f3 g7g6 d4e2 b8d7 e2g3 f6g7 c3d5 c6d5 f3d5 d7f6 d5f3 d8d1 f1d1 h7h5 c1e3 f6g4 b8d7 -> 3773 (V: 47.10%) (N: 5.49%) PV: b8d7 e5f6 d7f6 g2g4 h7h6 g4g5 h6g5 f4g5 f6e4 c3e4 d5e4 d4f5 c8f5 f1f5 d8d1 e2d1 a8d8 d1e2 g7g6 f5e5 e7d6 e5e4 f8e8 e4e8 f6e4 -> 1545 (V: 45.38%) (N: 37.81%) PV: f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f5d6 c6d4 e2d1 d8h4 c1e3 h7h5 g4h5 c8h3 e3d4 c5d4 f6d7 -> 594 (V: 46.07%) (N: 9.38%) PV: f6d7 d4f5 e7c5 g1h1 a7a5 c3a4 b7b5 a4c5 d7c5 f5d6 c8a6 f4f5 b8d7 c1f4 d8b6 e5e6 f7e6 f5e6 c5e6 d8b6 -> 433 (V: 47.03%) (N: 1.02%) PV: d8b6 e5f6 e7f6 c1e3 b6b2 c3a4 b2a3 e2f3 a3a4 d4b3 f6a1 d1a1 b8a6 a1d1 c8e6 a7a6 -> 150 (V: 45.60%) (N: 3.29%) PV: a7a6 e5f6 e7f6 d4f3 b8d7 f3g5 f6g5 f4g5 d7e5 c1f4 e5g6 f4g3 e7c5 -> 136 (V: 45.49%) (N: 3.18%) PV: e7c5 e5f6 d8f6 c1e3 f8e8 e3f2 f6f4 g2g3 f4f6 c3a4 c5d4 d1d4 e8e2 d4f6 g7f6 a1e1 h7h6 -> 128 (V: 44.95%) (N: 3.61%) PV: h7h6 e5f6 e7f6 d4f3 b8d7 a2a4 a7a5 g2g3 f8e8 e2d3 d7c5 f1e1 c5d3 c2d3 g7g6 -> 120 (V: 45.65%) (N: 2.48%) PV: g7g6 e5f6 e7f6 d4f3 f6g7 f3g5 h7h6 g5f3 f8e8 f3e5 b8d7 e5d7 c8d7 e2f3 d5d4 f6g4 -> 89 (V: 35.61%) (N: 10.03%) PV: f6g4 e2g4 c8g4 d1g4 g7g6 f4f5 b8d7 e5e6 d7e5 g4g3 e7h4 g3e5 d8f6 h7h5 -> 82 (V: 44.45%) (N: 2.65%) PV: h7h5 e5f6 e7f6 e2h5 g7g6 h5f3 f6d4 d1d4 b8a6 d4e5 f7f6 e5e2 f8e8 e2d1 c8f5 b8a6 -> 56 (V: 44.09%) (N: 2.04%) PV: b8a6 e5f6 e7f6 d4f3 a6c5 f3e5 f8e8 e2h5 g7g6 h5f3 f6e5 f4e5 e8e5 c1f4 e5f5 c6c5 -> 55 (V: 40.39%) (N: 3.83%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7 f3e5 b8c6 e5c6 b7c6 f4f5 h7h6 f5f6 f6e8 -> 43 (V: 44.36%) (N: 1.43%) PV: f6e8 f4f5 f7f6 e5e6 e7c5 c1e3 d8b6 c3a4 b6a5 a4c5 a5c5 e2h5 d8e8 -> 24 (V: 40.63%) (N: 1.61%) PV: d8e8 e5f6 e7f6 d4f3 b8a6 f3e5 f6e5 f4e5 e8e5 c1f4 e5f6 f4g3 f6g6 b7b5 -> 21 (V: 43.47%) (N: 0.92%) PV: b7b5 e5f6 e7f6 d4f3 b5b4 c3a4 f8e8 a4c5 a7a5 f3e5 b8d7 c5d7 c8d7 e5d7 f8e8 -> 17 (V: 41.76%) (N: 1.02%) PV: f8e8 e5f6 e7f6 d4f3 h7h6 f3e5 f6e5 f4e5 e8e5 c1f4 e5f5 e2g4 f5f6 g4c8 f6h5 -> 16 (V: 35.12%) (N: 1.94%) PV: f6h5 e2h5 g7g6 h5f3 e7c5 c1e3 d8h4 g2g3 h4h3 f3g2 h3h5 d1h5 g6h5 g8h8 -> 12 (V: 43.30%) (N: 0.54%) PV: g8h8 e5f6 e7f6 d4f3 h8g8 f3e5 b8d7 e5d7 c8d7 e2f3 d5d4 c3e4 f6e7 c8d7 -> 10 (V: 40.18%) (N: 0.69%) PV: c8d7 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 c1f4 e5e7 b7b6 -> 9 (V: 41.23%) (N: 0.61%) PV: b7b6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 c1f4 d8d7 -> 7 (V: 38.71%) (N: 0.63%) PV: d8d7 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e7b4 -> 5 (V: 37.66%) (N: 0.48%) PV: e7b4 e5f6 d8f6 d4f3 b4c3 b2c3 d8c7 -> 2 (V: 33.88%) (N: 0.31%) PV: d8c7 e5f6 e7f6 c8g4 -> 2 (V: 28.93%) (N: 0.47%) PV: c8g4 e2g4 f6g4 g7g5 -> 0 (V: 51.27%) (N: 0.22%) PV: g7g5 d8a5 -> 0 (V: 51.27%) (N: 0.21%) PV: d8a5 c8e6 -> 0 (V: 51.27%) (N: 0.18%) PV: c8e6 c8h3 -> 0 (V: 51.27%) (N: 0.18%) PV: c8h3 c8f5 -> 0 (V: 51.27%) (N: 0.13%) PV: c8f5 e7d6 -> 0 (V: 51.27%) (N: 0.10%) PV: e7d6 e7a3 -> 0 (V: 51.27%) (N: 0.10%) PV: e7a3 d8d6 -> 0 (V: 51.27%) (N: 0.05%) PV: d8d6 info depth 22 nodes 20000 nps 175 score cp -66 winrate 47.45% time 114244 pv a7a5 e5f6 e7f6 e2f3 g7g6 d4e2 b8d7 e2g3 f6g7 c3d5 c6d5 f3d5 d7f6 d5f3 d8d1 f1d1 h7h5 c1e3 f6g4 bestmove a7a5 ```

Result is the same, only here Leela first switches to Nbd7, and then to a5.

..a5 It's an almost impossible movement to get a good result xDDD The model simply misevaluate in some point the game state. Problem is that maybe we are a little bit overvaluing the real ELO gained by the model. No real ELO greater than 1200 would do such a blunder.

I have a suggestion about this taking into account that the problem only seems to happen when we change the number of simulations (as @jkiliani has shown):

As we are doing match for promoting always in 800 simulations during self-training, don't you think this kind of evolutionary selection by tournament would promote in time those networks best adapted to play well in 800 simulations?

In that case changing the number of simulation (even raising up) could affect the game strength since we are changing the environment and the model could not be as good in that new situation. what do you think @Error323 @jkiliani @glinscott ?

In fact I saw this kind of things in my own repositories, even in the version of Connect4 I did:

https://github.com/Zeta36/chess-alpha-zero https://github.com/Zeta36/connect4-alpha-zero

Maybe DeepMind always promoted in AZ because of this problem of self-adaptation due to the tournament environment (we'll have to wait for the Nature paper xDD)

For me, the most likely suspect is that we're dealing with model overfitting here, since the search converges on a bad move at high playouts. Could @Error323 please post the Tensorboard stats for Id 43? If this hypothesis is confirmed, we will need either symmetries implemented, or if that's not possible, another increase to the training window soon.

Well, that's a possibility @jkiliani, but it could also be a self-adaptation to the match promoting conditions (in this case 800 simulations).

If you make networks to fight against each other in a determined tournament, logically this evolutionary tournament process will select in time the networks best adapted to that precise conditions.

In summary: we are selecting networks doing a evolutionary pressure towards NN that plays as best as possible in a 800 simulations MTCS environment. So the networks weights would be also conditioned to that kind of characteristic environment.

And if you change after training the environment (number of simulations in this case), the weights of the NN will start to calculate in strange unseen situations for it.

One solution would be not doing anything and finally always playing with 800 simulations (maybe DeepMind did this after convergence). Another solution would be make the promotion matches in the desired final situation: 1200 simulations or whatever.

I repeated this experiment with three more networks at 20k playouts:

e388c9 (Id 38)

``` lczero -w ~/Downloads/LCZero-Networks/e388c9 -p 20000 -t1 --noponder Using 1 thread(s). Generated 1924 moves Detecting residual layers...v1...64 channels...6 blocks. position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 go RNG seed: 0xe9a213920be30685 (thread: 10181866982453914292) info depth 7 nodes 1 nps 58 score cp -87 winrate 46.65% time 16 pv f6e4 c3e4 info depth 8 nodes 3 nps 96 score cp -136 winrate 44.78% time 30 pv f6e4 c3e4 d5e4 info depth 9 nodes 5 nps 100 score cp -124 winrate 45.21% time 49 pv f6e4 c3e4 d5e4 f4f5 info depth 10 nodes 9 nps 112 score cp -138 winrate 44.67% time 79 pv f6e4 c3e4 d5e4 f4f5 c6c5 info depth 11 nodes 17 nps 125 score cp -133 winrate 44.90% time 134 pv f6e4 c3e4 d5e4 c2c3 c6c5 d4b5 info depth 12 nodes 33 nps 138 score cp -96 winrate 46.31% time 238 pv f6e4 c3e4 d5e4 c2c3 c6c5 d4b5 a7a6 info depth 13 nodes 59 nps 147 score cp -95 winrate 46.33% time 398 pv f6e4 c3e4 d5e4 c2c3 c6c5 d4b5 a7a6 info depth 14 nodes 106 nps 157 score cp -91 winrate 46.49% time 674 pv f6e4 c3e4 d5e4 c2c3 c6c5 d4b5 a7a6 b5d6 info depth 15 nodes 189 nps 156 score cp -76 winrate 47.05% time 1203 pv f6e4 c3e4 d5e4 c2c3 c6c5 d4b5 a7a6 b5d6 info depth 16 nodes 349 nps 156 score cp -56 winrate 47.83% time 2223 pv f6e4 c3e4 d5e4 c2c3 c6c5 d4b5 a7a6 b5d6 b8c6 info depth 17 nodes 636 nps 158 score cp -46 winrate 48.22% time 4021 pv f6e4 c3e4 d5e4 c2c3 c6c5 d4b5 a7a6 b5d6 b8c6 d6e4 info depth 18 nodes 1162 nps 150 score cp -41 winrate 48.42% time 7725 pv f6e4 c3e4 d5e4 f4f5 c6c5 d4b5 b8c6 c1f4 c8f5 b5d6 info depth 19 nodes 2088 nps 153 score cp -44 winrate 48.30% time 13640 pv f6e4 c3e4 d5e4 f4f5 c6c5 d4b5 a7a6 b5d6 b8c6 f5f6 info depth 20 nodes 3760 nps 155 score cp -42 winrate 48.38% time 24139 pv f6d7 d4f5 e7c5 g1h1 g7g6 f5h6 g8g7 h6g4 d8h4 g2g3 h4h3 info depth 21 nodes 6835 nps 160 score cp -29 winrate 48.87% time 42689 pv f6d7 d4f5 e7c5 g1h1 g7g6 f5h6 g8g7 h6g4 h7h5 g4f6 d7f6 info depth 22 nodes 12348 nps 159 score cp -22 winrate 49.14% time 77561 pv f6d7 a2a4 e7c5 c1e3 a7a6 e2f3 g7g6 a4a5 f6d7 -> 14936 (V: 49.71%) (N: 8.66%) PV: f6d7 a2a4 a7a6 d4f5 e7c5 g1h1 g7g6 f5h6 g8g7 h6g4 d5d4 c3e4 f7f5 f6e4 -> 1829 (V: 47.83%) (N: 29.65%) PV: f6e4 c3e4 d5e4 f4f5 c6c5 d4b5 b8c6 f5f6 e7f6 e5f6 g7f6 d1d8 b8d7 -> 1047 (V: 49.21%) (N: 4.95%) PV: b8d7 e5f6 d7f6 g2g4 h7h6 g4g5 h6g5 f4g5 f6e4 c3e4 d5e4 c2c3 a7a6 -> 403 (V: 48.99%) (N: 2.57%) PV: a7a6 e5f6 e7f6 d4f3 b8d7 a2a4 d7c5 a4a5 a7a5 -> 305 (V: 48.02%) (N: 4.45%) PV: a7a5 e5f6 e7f6 d4f3 b8d7 a2a4 d7c5 f3e5 h7h6 -> 281 (V: 48.33%) (N: 3.35%) PV: h7h6 e5f6 e7f6 d4f3 a7a5 a2a4 b8a6 f3e5 d8e8 -> 217 (V: 49.12%) (N: 1.15%) PV: d8e8 e5f6 e7f6 d4f3 h7h6 f3e5 b8d7 e5d7 f6g4 -> 150 (V: 35.92%) (N: 17.39%) PV: f6g4 e2g4 c8g4 d1g4 g7g6 f4f5 b8d7 e5e6 d7e5 g4g3 b8a6 -> 130 (V: 47.39%) (N: 2.59%) PV: b8a6 e5f6 e7f6 d4f3 f8e8 a2a4 d5d4 g7g6 -> 120 (V: 47.07%) (N: 2.72%) PV: g7g6 e5f6 e7f6 d4f3 f6g7 a2a4 b8a6 f3e5 f8e8 -> 108 (V: 48.91%) (N: 0.78%) PV: f8e8 e5f6 e7f6 d4f3 d5d4 c3a4 b7b5 c6c5 -> 74 (V: 42.45%) (N: 4.38%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7 f3e5 b8c6 f6e8 -> 73 (V: 47.40%) (N: 1.45%) PV: f6e8 f4f5 b8d7 e5e6 d7e5 c1f4 d8b6 -> 70 (V: 48.37%) (N: 0.82%) PV: d8b6 e5f6 e7f6 c1e3 b6b2 c3a4 b2a3 c2c3 h7h5 -> 61 (V: 42.41%) (N: 3.79%) PV: h7h5 e5f6 e7f6 e2h5 g7g6 h5f3 f6d4 d1d4 f8e8 e7c5 -> 47 (V: 43.76%) (N: 2.40%) PV: e7c5 e5f6 d8f6 c1e3 f6e7 e3f2 b8a6 c8d7 -> 39 (V: 47.44%) (N: 0.68%) PV: c8d7 e5f6 e7f6 d4f3 b8a6 f3e5 f6e5 f4e5 d8h4 b7b5 -> 34 (V: 45.56%) (N: 1.23%) PV: b7b5 e5f6 e7f6 d4f3 b5b4 c3a4 a7a5 c2c3 f6h5 -> 16 (V: 35.10%) (N: 2.05%) PV: f6h5 e2h5 g7g6 h5f3 e7c5 c1e3 d8h4 b7b6 -> 13 (V: 42.71%) (N: 0.77%) PV: b7b6 e5f6 e7f6 d4f3 a7a6 f3e5 f6e5 d8d7 -> 12 (V: 43.49%) (N: 0.64%) PV: d8d7 e5f6 e7f6 d4f3 d7d8 f3e5 f6e5 c8g4 -> 9 (V: 37.04%) (N: 0.94%) PV: c8g4 e2g4 f6g4 d1g4 g7g6 f4f5 b8d7 g8h8 -> 7 (V: 40.34%) (N: 0.53%) PV: g8h8 e5f6 e7f6 d4f3 h8g8 f3e5 c8e6 -> 4 (V: 41.45%) (N: 0.34%) PV: c8e6 e5f6 e7f6 d4e6 f7e6 e7b4 -> 4 (V: 37.18%) (N: 0.49%) PV: e7b4 e5f6 d8f6 d4f3 d8a5 -> 3 (V: 43.10%) (N: 0.16%) PV: d8a5 e5f6 e7f6 d4b3 d8c7 -> 2 (V: 38.60%) (N: 0.28%) PV: d8c7 e5f6 e7f6 c8f5 -> 1 (V: 39.65%) (N: 0.11%) PV: c8f5 d4f5 e7d6 -> 1 (V: 38.46%) (N: 0.08%) PV: e7d6 e5d6 c8h3 -> 1 (V: 37.19%) (N: 0.17%) PV: c8h3 e5f6 e7a3 -> 1 (V: 35.33%) (N: 0.10%) PV: e7a3 e5f6 g7g5 -> 1 (V: 32.58%) (N: 0.26%) PV: g7g5 f4g5 d8d6 -> 1 (V: 22.83%) (N: 0.06%) PV: d8d6 e5d6 info depth 22 nodes 20000 nps 166 score cp -21 winrate 49.18% time 119922 pv f6d7 a2a4 a7a6 d4f5 e7c5 g1h1 g7g6 f5h6 g8g7 h6g4 d5d4 c3e4 f7f5 bestmove f6d7 ```

fde903 (Id 41)

``` lczero -w ~/Downloads/LCZero-Networks/fde903 -p 20000 -t1 --noponder Using 1 thread(s). Generated 1924 moves Detecting residual layers...v1...64 channels...6 blocks. position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 go RNG seed: 0x37c93f9e6e189191 (thread: 10181866982453914292) info depth 7 nodes 1 nps 76 score cp 2 winrate 50.09% time 12 pv f6e4 c3e4 info depth 8 nodes 3 nps 115 score cp -61 winrate 47.62% time 25 pv f6e4 c3e4 d5e4 info depth 9 nodes 5 nps 131 score cp -25 winrate 49.03% time 37 pv f6e4 c3e4 d5e4 f4f5 info depth 10 nodes 10 nps 129 score cp -69 winrate 47.32% time 76 pv f6e4 c3e4 d5e4 c2c3 c6c5 info depth 11 nodes 17 nps 142 score cp -55 winrate 47.85% time 118 pv f6e4 c3e4 d5e4 c2c3 c6c5 d4b5 info depth 12 nodes 32 nps 142 score cp -43 winrate 48.31% time 224 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 info depth 13 nodes 59 nps 147 score cp -46 winrate 48.23% time 399 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 info depth 14 nodes 106 nps 148 score cp -59 winrate 47.70% time 714 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 b5d6 info depth 15 nodes 192 nps 150 score cp -49 winrate 48.09% time 1277 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 b5d6 info depth 16 nodes 350 nps 152 score cp -17 winrate 49.33% time 2298 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 b5d6 e7d6 info depth 17 nodes 643 nps 154 score cp -10 winrate 49.58% time 4171 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5c3 b8c6 info depth 18 nodes 1158 nps 154 score cp -16 winrate 49.38% time 7494 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5c3 b8c6 info depth 19 nodes 2089 nps 155 score cp -17 winrate 49.33% time 13423 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 b5d6 e7d6 info depth 20 nodes 3760 nps 159 score cp -12 winrate 49.52% time 23592 pv f6d7 d4f5 e7c5 g1h1 a7a5 c3a4 b7b5 a4c5 d7c5 info depth 21 nodes 6825 nps 164 score cp -1 winrate 49.95% time 41397 pv f6d7 d4f5 e7c5 g1h1 g7g6 f5h6 g8g7 h6g4 f7f5 g4e3 d5d4 c3a4 info depth 22 nodes 12266 nps 166 score cp -6 winrate 49.74% time 73870 pv f6d7 g2g4 h7h6 d4f5 e7c5 g1g2 a7a5 e2f3 g7g6 f5h6 f6d7 -> 12286 (V: 49.69%) (N: 8.90%) PV: f6d7 g2g3 a7a5 d4f5 e7c5 g1g2 g7g6 f5h6 g8g7 h6g4 h7h5 g4e3 d5d4 c3e4 f6e4 -> 3441 (V: 48.66%) (N: 31.25%) PV: f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f5d6 c6d4 e2d1 d8h4 c2c3 d4f3 b8d7 -> 790 (V: 49.01%) (N: 4.89%) PV: b8d7 e5f6 d7f6 g2g4 h7h6 g4g5 h6g5 f4g5 f6e4 c3e4 d5e4 a7a5 -> 629 (V: 49.11%) (N: 3.29%) PV: a7a5 e5f6 e7f6 d4f3 b8d7 g2g3 f8e8 a2a4 h7h6 -> 391 (V: 48.50%) (N: 4.03%) PV: h7h6 e5f6 e7f6 d4f3 f8e8 f3e5 f6e5 f4e5 e8e5 a7a6 -> 373 (V: 48.69%) (N: 3.28%) PV: a7a6 e5f6 e7f6 g2g4 h7h6 d4f5 b8d7 g4g5 g7g6 -> 360 (V: 48.73%) (N: 3.02%) PV: g7g6 e5f6 e7f6 d4f3 f6g7 a2a4 b8a6 a4a5 b8a6 -> 301 (V: 48.72%) (N: 2.58%) PV: b8a6 e5f6 e7f6 d4f3 f8e8 a2a3 h7h6 d8b6 -> 300 (V: 49.31%) (N: 1.02%) PV: d8b6 e5f6 e7f6 c1e3 f8e8 e3f2 b8d7 c3a4 b6d8 d8e8 -> 219 (V: 48.90%) (N: 1.52%) PV: d8e8 e5f6 e7f6 e2f3 f6d4 d1d4 f7f6 d4d1 b8a6 h7h5 -> 170 (V: 47.28%) (N: 3.53%) PV: h7h5 e5f6 e7f6 e2h5 g7g6 h5f3 f6g7 c3e2 f8e8 f6g4 -> 142 (V: 38.30%) (N: 13.63%) PV: f6g4 e2g4 h7h6 g4c8 d8c8 f4f5 b8d7 e5e6 b7b5 -> 129 (V: 48.50%) (N: 1.32%) PV: b7b5 e5f6 e7f6 d4f3 b5b4 c3a4 a7a5 c2c3 f8e8 f8e8 -> 124 (V: 48.60%) (N: 1.07%) PV: f8e8 e5f6 e7f6 d4f3 h7h6 a2a4 d5d4 c3e4 c6c5 -> 78 (V: 43.03%) (N: 4.38%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7 f3e5 b8c6 e5c6 f6e8 -> 58 (V: 46.65%) (N: 1.46%) PV: f6e8 f4f5 f7f6 e5e6 g7g6 e7c5 -> 58 (V: 42.97%) (N: 3.32%) PV: e7c5 e5f6 d8f6 c1e3 f6e7 e3f2 b8d7 c8d7 -> 52 (V: 48.14%) (N: 0.71%) PV: c8d7 e5f6 e7f6 d4f3 h7h6 f3e5 f6e5 f4e5 d8h4 d8d7 -> 22 (V: 45.78%) (N: 0.61%) PV: d8d7 e5f6 e7f6 d4f3 d7d8 f3e5 b8d7 f6h5 -> 22 (V: 38.11%) (N: 2.17%) PV: f6h5 e2h5 g7g6 h5f3 e7c5 c3e2 d8h4 b7b6 -> 18 (V: 44.25%) (N: 0.70%) PV: b7b6 e5f6 e7f6 d4f3 h7h6 f3e5 f6e5 g8h8 -> 10 (V: 43.86%) (N: 0.51%) PV: g8h8 e5f6 e7f6 d4f3 h8g8 f3e5 b8d7 e5d7 c8g4 -> 6 (V: 38.34%) (N: 0.65%) PV: c8g4 e2g4 f6g4 d1g4 g7g6 e7b4 -> 6 (V: 37.45%) (N: 0.65%) PV: e7b4 e5f6 d8f6 a2a3 b4e7 c8e6 -> 4 (V: 41.11%) (N: 0.20%) PV: c8e6 e5f6 e7f6 d4e6 f7e6 d8c7 -> 3 (V: 38.89%) (N: 0.33%) PV: d8c7 e5f6 e7f6 d4f3 d8a5 -> 2 (V: 38.96%) (N: 0.18%) PV: d8a5 e5f6 e7f6 c8f5 -> 1 (V: 40.46%) (N: 0.13%) PV: c8f5 d4f5 e7d6 -> 1 (V: 37.28%) (N: 0.09%) PV: e7d6 e5d6 g7g5 -> 1 (V: 31.35%) (N: 0.30%) PV: g7g5 e5f6 e7a3 -> 1 (V: 29.85%) (N: 0.12%) PV: e7a3 e5f6 c8h3 -> 1 (V: 28.90%) (N: 0.15%) PV: c8h3 e5f6 d8d6 -> 1 (V: 20.79%) (N: 0.05%) PV: d8d6 e5d6 info depth 22 nodes 20000 nps 160 score cp -21 winrate 49.17% time 124254 pv f6d7 g2g3 a7a5 d4f5 e7c5 g1g2 g7g6 f5h6 g8g7 h6g4 h7h5 g4e3 d5d4 c3e4 bestmove f6d7 ```

d9e0d1 (Id 42)

``` lczero -w ~/Downloads/LCZero-Networks/d9e0d1 -p 20000 -t1 --noponder Using 1 thread(s). Generated 1924 moves Detecting residual layers...v1...64 channels...6 blocks. position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 go RNG seed: 0xf48f6b690f45fdf8 (thread: 10181866982453914292) info depth 7 nodes 1 nps 71 score cp -192 winrate 42.66% time 13 pv f6e4 c3e4 info depth 8 nodes 3 nps 115 score cp -140 winrate 44.63% time 25 pv f6e4 c3e4 d5e4 info depth 9 nodes 5 nps 128 score cp -95 winrate 46.32% time 38 pv f6e4 c3e4 d5e4 c2c3 c6c5 info depth 10 nodes 10 nps 128 score cp -83 winrate 46.78% time 77 pv f6e4 c3e4 d5e4 f4f5 c6c5 d4b5 info depth 11 nodes 17 nps 133 score cp -95 winrate 46.34% time 126 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 info depth 12 nodes 31 nps 131 score cp -129 winrate 45.05% time 234 pv f6e4 c3e4 d5e4 c1e3 g7g6 f4f5 info depth 13 nodes 57 nps 145 score cp -92 winrate 46.44% time 390 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b3 d8d1 info depth 14 nodes 101 nps 152 score cp -79 winrate 46.95% time 662 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5d6 info depth 15 nodes 186 nps 152 score cp -68 winrate 47.38% time 1219 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5d6 info depth 16 nodes 337 nps 153 score cp -54 winrate 47.91% time 2194 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5c3 b8c6 info depth 17 nodes 615 nps 153 score cp -51 winrate 48.04% time 4018 pv f6e4 c3e4 d5e4 c1e3 d8b6 f4f5 b6b2 f5f6 e7f6 info depth 18 nodes 1130 nps 155 score cp -48 winrate 48.15% time 7276 pv f6e4 c3e4 d5e4 f4f5 c6c5 f5f6 c5d4 f6e7 d8e7 d1d4 b8c6 d4e4 f7f5 e4f3 info depth 19 nodes 2049 nps 154 score cp -57 winrate 47.80% time 13268 pv f6e4 c3e4 d5e4 f4f5 e7c5 c1e3 b8d7 e5e6 d7e5 c2c3 info depth 20 nodes 3718 nps 153 score cp -54 winrate 47.92% time 24288 pv f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f5d6 c6d4 e2d1 info depth 21 nodes 6716 nps 157 score cp -71 winrate 47.26% time 42726 pv f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f1d1 g7g6 f5d6 c6d4 e2c4 info depth 22 nodes 12221 nps 161 score cp -51 winrate 48.01% time 75664 pv f6d7 d4f5 e7c5 g1h1 a7a5 c1e3 c5e3 f5e3 d7b6 e2f3 d5d4 f6d7 -> 12354 (V: 49.13%) (N: 5.72%) PV: f6d7 d4f5 e7c5 c1e3 c5e3 f5e3 d7c5 b2b4 c5e6 e3f5 a7a5 b4a5 a8a5 f5d6 f6e4 -> 4238 (V: 47.41%) (N: 42.31%) PV: f6e4 c3e4 d5e4 g2g4 c6c5 d4f5 b8c6 d1d8 e7d8 f1d1 g7g6 f5d6 c6d4 e2c4 d8b6 -> 924 (V: 48.07%) (N: 1.02%) PV: d8b6 c3a4 b6d8 e5f6 e7f6 c2c3 b8d7 f4f5 d7e5 c1f4 b8d7 -> 708 (V: 47.79%) (N: 3.56%) PV: b8d7 e5f6 d7f6 g2g4 h7h6 f4f5 e7c5 c1e3 a7a5 -> 403 (V: 47.67%) (N: 2.57%) PV: a7a5 e5f6 e7f6 d4f3 b8d7 g2g3 f8e8 a2a4 a7a6 -> 255 (V: 47.41%) (N: 2.53%) PV: a7a6 e5f6 e7f6 d4f3 b8d7 f4f5 f8e8 g7g6 -> 229 (V: 47.36%) (N: 2.39%) PV: g7g6 e5f6 e7f6 d4f3 b8d7 c3d5 c6d5 b8a6 -> 155 (V: 47.20%) (N: 1.90%) PV: b8a6 e5f6 e7f6 d4f3 f8e8 a2a4 d5d4 d8e8 -> 146 (V: 47.55%) (N: 1.19%) PV: d8e8 e5f6 e7f6 d4f3 b8d7 a2a4 a7a6 f6g4 -> 142 (V: 36.05%) (N: 15.59%) PV: f6g4 e2g4 c8g4 d1g4 e7c5 c1e3 g7g6 f4f5 h7h6 -> 140 (V: 46.40%) (N: 3.16%) PV: h7h6 e5f6 e7f6 d4f3 f8e8 a2a4 d5d4 c3e4 h7h5 -> 67 (V: 44.41%) (N: 2.60%) PV: h7h5 e5f6 e7f6 e2h5 g7g6 h5f3 f6d4 d1d4 f8e8 c6c5 -> 46 (V: 40.20%) (N: 3.50%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7 f3e5 b8c6 e7c5 -> 44 (V: 41.01%) (N: 2.98%) PV: e7c5 e5f6 d8f6 c1e3 f6e7 e3f2 f6e8 -> 39 (V: 45.89%) (N: 1.04%) PV: f6e8 f4f5 f7f6 e5e6 g7g6 b7b5 -> 28 (V: 45.39%) (N: 0.90%) PV: b7b5 e5f6 e7f6 d4f3 b5b4 c3a4 a7a5 f8e8 -> 22 (V: 45.42%) (N: 0.71%) PV: f8e8 e5f6 e7f6 d4f3 h7h6 f3e5 f6e5 f6h5 -> 14 (V: 36.55%) (N: 1.54%) PV: f6h5 e2h5 g7g6 h5f3 e7c5 c1e3 d8h4 c8d7 -> 11 (V: 43.13%) (N: 0.59%) PV: c8d7 e5f6 e7f6 d4f3 b8a6 f3e5 f6e5 c8g4 -> 7 (V: 35.58%) (N: 0.85%) PV: c8g4 e2g4 f6g4 d1g4 g7g6 f4f5 d8d7 -> 6 (V: 39.32%) (N: 0.57%) PV: d8d7 e5f6 e7f6 d4f3 h7h6 f3e5 g8h8 -> 5 (V: 38.18%) (N: 0.46%) PV: g8h8 e5f6 e7f6 d4f3 h7h6 f3e5 e7b4 -> 4 (V: 33.89%) (N: 0.50%) PV: e7b4 e5f6 d8f6 d4f3 b7b6 -> 3 (V: 33.94%) (N: 0.49%) PV: b7b6 e5f6 e7f6 d4f3 d8c7 -> 2 (V: 34.02%) (N: 0.31%) PV: d8c7 e5f6 e7f6 c8e6 -> 1 (V: 38.76%) (N: 0.14%) PV: c8e6 e5f6 d8a5 -> 1 (V: 36.91%) (N: 0.18%) PV: d8a5 e5f6 c8f5 -> 1 (V: 35.56%) (N: 0.11%) PV: c8f5 d4f5 e7d6 -> 1 (V: 32.39%) (N: 0.09%) PV: e7d6 e5d6 c8h3 -> 1 (V: 28.69%) (N: 0.16%) PV: c8h3 e5f6 e7a3 -> 1 (V: 26.52%) (N: 0.09%) PV: e7a3 e5f6 g7g5 -> 1 (V: 25.73%) (N: 0.20%) PV: g7g5 e5f6 d8d6 -> 1 (V: 14.24%) (N: 0.05%) PV: d8d6 e5d6 info depth 22 nodes 20000 nps 162 score cp -42 winrate 48.35% time 123031 pv f6d7 d4f5 e7c5 c1e3 c5e3 f5e3 d7c5 b2b4 c5e6 e3f5 a7a5 b4a5 a8a5 f5d6 bestmove f6d7 ```

Summary: Id 38, Id 41 and Id 42 ALL play Nfd7. All of them initially plan on Ne4 and switch to Nfd7 at higher playouts. So this problem is not with Leela Chess in general or the search code, but with Id 43. I really think we should look into overfitting here.

Edit: I just added Id 44 (c2fb7b), it also plays Nfd7 after spending being on Ne4 for much of the search:

c2fb7b (Id 44)

``` lczero -w ~/Downloads/LCZero-Networks/c2fb7b -p 20000 -t1 --noponder Using 1 thread(s). Generated 1924 moves Detecting residual layers...v1...64 channels...6 blocks. position startpos moves e2e4 e7e5 g1f3 d7d6 d2d4 e5d4 f3d4 g8f6 b1c3 c7c6 f1e2 f8e7 e1g1 e8g8 f2f4 d6d5 e4e5 go RNG seed: 0x97bc318730244fa3 (thread: 10181866982453914292) info depth 7 nodes 1 nps 62 score cp 96 winrate 53.69% time 15 pv f6e4 c3e4 info depth 8 nodes 3 nps 88 score cp 54 winrate 52.11% time 33 pv f6e4 c3e4 d5e4 info depth 9 nodes 5 nps 102 score cp 48 winrate 51.88% time 48 pv f6e4 c3e4 d5e4 c2c3 c6c5 info depth 10 nodes 10 nps 117 score cp 4 winrate 50.16% time 84 pv f6e4 c3e4 d5e4 f4f5 c6c5 d4b5 info depth 11 nodes 17 nps 128 score cp -33 winrate 48.73% time 131 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 info depth 12 nodes 31 nps 134 score cp -59 winrate 47.71% time 229 pv f6e4 c3e4 d5e4 c1e3 g7g6 f4f5 info depth 13 nodes 57 nps 146 score cp -62 winrate 47.59% time 389 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 info depth 14 nodes 105 nps 147 score cp -48 winrate 48.13% time 710 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 b8c6 d1d8 info depth 15 nodes 189 nps 149 score cp -23 winrate 49.11% time 1262 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5d6 info depth 16 nodes 341 nps 152 score cp -35 winrate 48.63% time 2239 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 d8b6 b5d6 info depth 17 nodes 619 nps 155 score cp -30 winrate 48.84% time 3980 pv f6e4 c3e4 d5e4 c1e3 c6c5 d4b5 a7a6 b5c3 b8c6 info depth 18 nodes 1132 nps 153 score cp -33 winrate 48.73% time 7384 pv f6e4 c3e4 d5e4 f4f5 c6c5 f5f6 c5d4 f6e7 d8e7 d1d4 b8c6 d4e4 f7f5 e4f3 info depth 19 nodes 2027 nps 154 score cp -22 winrate 49.14% time 13157 pv f6e4 c3e4 d5e4 f4f5 c6c5 d4b5 b8c6 f5f6 e7f6 e5f6 d8f6 f1f6 info depth 20 nodes 3692 nps 153 score cp -32 winrate 48.77% time 23977 pv f6e4 c3e4 d5e4 f4f5 c6c5 d4b3 d8d1 e2d1 b8c6 f5f6 e7f6 e5f6 g7f6 c1h6 info depth 21 nodes 6724 nps 154 score cp -41 winrate 48.40% time 43635 pv f6e4 c3e4 d5e4 f4f5 c6c5 d4b3 d8d1 e2d1 b8c6 f5f6 e7f6 e5f6 g7f6 c1h6 f8d8 info depth 22 nodes 12184 nps 160 score cp -23 winrate 49.11% time 75832 pv f6d7 d4f5 e7c5 g1h1 d7b6 f5d6 c5d6 e5d6 d8d6 a2a4 b8a6 a4a5 b6d7 f6d7 -> 13834 (V: 49.49%) (N: 9.32%) PV: f6d7 d4f5 e7c5 c1e3 c5e3 f5e3 d7c5 e2g4 d5d4 g4c8 d8c8 d1d4 c5e6 d4d3 b8a6 f6e4 -> 2668 (V: 47.80%) (N: 39.34%) PV: f6e4 c3e4 d5e4 f4f5 c6c5 d4b3 d8d1 e2d1 b8c6 f5f6 e7f6 e5f6 c5c4 f6g7 g8g7 b8d7 -> 1050 (V: 49.11%) (N: 4.06%) PV: b8d7 e5f6 d7f6 g2g4 h7h6 f4f5 e7c5 c1e3 f8e8 a7a5 -> 520 (V: 48.96%) (N: 2.56%) PV: a7a5 e5f6 e7f6 e2f3 f6d4 d1d4 b8a6 c1e3 a6b4 a2a3 d8b6 -> 427 (V: 49.29%) (N: 0.93%) PV: d8b6 e5f6 e7f6 c1e3 f8e8 e3f2 b6b2 c3a4 b2b4 g7g6 -> 289 (V: 48.74%) (N: 1.98%) PV: g7g6 e5f6 e7f6 f4f5 f6g7 f5g6 h7g6 d4f3 b8d7 f3g5 h7h6 -> 275 (V: 48.17%) (N: 3.20%) PV: h7h6 e5f6 e7f6 d4f3 b8d7 f4f5 f8e8 a7a6 -> 221 (V: 48.05%) (N: 2.78%) PV: a7a6 e5f6 e7f6 d4f3 b8d7 f4f5 f8e8 b8a6 -> 193 (V: 48.53%) (N: 1.63%) PV: b8a6 e5f6 e7f6 d4f3 a6c5 f3e5 d5d4 c3e4 f6g4 -> 149 (V: 36.77%) (N: 15.89%) PV: f6g4 e2g4 c8g4 d1g4 e7c5 c1e3 g7g6 f4f5 h7h5 -> 71 (V: 45.79%) (N: 2.25%) PV: h7h5 e5f6 e7f6 e2h5 g7g6 h5f3 f6d4 d1d4 f8e8 d8e8 -> 63 (V: 47.20%) (N: 1.26%) PV: d8e8 e5f6 e7f6 d4f3 b8a6 a2a3 a6c5 c6c5 -> 49 (V: 42.05%) (N: 3.11%) PV: c6c5 e5f6 e7f6 d4f3 d5d4 c3e4 f6e7 f3e5 b8c6 e7c5 -> 44 (V: 41.36%) (N: 2.84%) PV: e7c5 e5f6 d8f6 c1e3 f6e7 e3f2 b8d7 f6e8 -> 40 (V: 45.27%) (N: 1.46%) PV: f6e8 f4f5 b8d7 e5e6 d7e5 f6h5 -> 24 (V: 40.42%) (N: 1.89%) PV: f6h5 e2h5 g7g6 h5f3 e7c5 c1e3 b8a6 f8e8 -> 23 (V: 45.41%) (N: 0.75%) PV: f8e8 e5f6 e7f6 d4f3 h7h6 f3e5 f6e5 b7b5 -> 14 (V: 44.22%) (N: 0.67%) PV: b7b5 e5f6 e7f6 f4f5 b5b4 c3a4 c8d7 -> 13 (V: 43.93%) (N: 0.62%) PV: c8d7 e5f6 e7f6 d4f3 b8a6 f3e5 f6e5 g8h8 -> 5 (V: 41.13%) (N: 0.41%) PV: g8h8 e5f6 e7f6 d4f3 h8g8 f3e5 b7b6 -> 5 (V: 39.75%) (N: 0.45%) PV: b7b6 e5f6 e7f6 d4f3 h7h6 c8g4 -> 5 (V: 35.30%) (N: 0.59%) PV: c8g4 e5f6 e7f6 e2g4 d8d7 -> 4 (V: 38.49%) (N: 0.43%) PV: d8d7 e5f6 e7f6 d4f3 h7h6 e7b4 -> 4 (V: 36.99%) (N: 0.40%) PV: e7b4 e5f6 d8f6 d4f3 d8c7 -> 2 (V: 34.63%) (N: 0.26%) PV: d8c7 e5f6 e7f6 c8e6 -> 1 (V: 37.89%) (N: 0.14%) PV: c8e6 e5f6 d8a5 -> 1 (V: 36.26%) (N: 0.15%) PV: d8a5 e5f6 c8f5 -> 1 (V: 36.18%) (N: 0.11%) PV: c8f5 d4f5 c8h3 -> 1 (V: 35.24%) (N: 0.15%) PV: c8h3 e5f6 e7d6 -> 1 (V: 34.27%) (N: 0.08%) PV: e7d6 e5d6 g7g5 -> 1 (V: 28.91%) (N: 0.16%) PV: g7g5 e5f6 e7a3 -> 1 (V: 27.04%) (N: 0.08%) PV: e7a3 e5f6 d8d6 -> 1 (V: 14.88%) (N: 0.04%) PV: d8d6 e5d6 info depth 22 nodes 20000 nps 162 score cp -26 winrate 48.97% time 123064 pv f6d7 d4f5 e7c5 c1e3 c5e3 f5e3 d7c5 e2g4 d5d4 g4c8 d8c8 d1d4 c5e6 d4d3 b8a6 bestmove f6d7 ```

You simply found a weak spot of the Id 43 net, but it doesn't look persistent or anything to me.

but have you checked there is no blunders in others positions? even if the ..a5 is passed in other networks, still there could be blunders in other positions in high simulations (above 800).

You could do a match between same networks just with different simulations configured.

While I could run a simulation for every black move, it would help if you specify where the other blunders are. To me it looks like the black game is basically lost after ...a5 anyway.

The whole game is here: https://lichess.org/nzvZOugM

There are a lot of blunders. It lost a pawn, the knight, etc. Look into the game, @jkiliani.

I already did look into the game, but I'm not a strong chess player, to me the knight was the only really obvious blunder. I looked into this to help you out, but you can analyse the positions with lczero in the command line yourself if you want additional information but can't provide specific requests.

If you see everything ok, it'll be ok ;).

Anyway I think a match of 400 games between the same network playing against itself with different simulations number would be interesting to know how LC0 scale with this change in the number of simulations.

I agree, but I don't have a machine free to do matches like this at the moment, since I'm running other tests already. Maybe someone will pick this up...

@jkiliani you were right the network seems to scale fine with the number of simulations. Here is a match somebody share with me in the chat:

I close this issue.

glinscott / leela-chess

Possible UCI command bug (or model bug)? #192

I paste over here the thinking history: