Training progress gen3

[Error323]

This time I properly interleaved the chunks and we're back to our trusty sudden drop in MSE Loss. I made 4 passes through 99'080 chunks, resulting in 4'954'000 datapoints. Here are the results of two separate 50 game matches of gen3 vs gen2. And I also played it against gen1 for good measure.

Score of lc_gen3 vs lc_gen2: 48 - 1 - 1  [0.970] 50
Elo difference: 603.86 +/- nan

Score of lc_gen3 vs lc_gen2: 43 - 5 - 2  [0.880] 50
Elo difference: 346.12 +/- 172.18

Score of lc_gen3 vs lc_seed: 48 - 0 - 2  [0.980] 50
Elo difference: 676.08 +/- nan

The network is uploaded and machines are crunching towards the next generation.

[Error323]

Generation 6:

Score of lc_gen6 vs lc_gen5: 86 - 14 - 0  [0.860] 100
Elo difference: 315.35 +/- 105.75

Trained in 28K steps, bootstrapped from gen4 (not a typo, it was a test), using 100K chunks from gen4 and 5.

• lr 0.01 steps 0 ... 25K
• lr 0.001 steps 25K+1 ... 28K

From this I conclude that a 100K window of the previous 2 networks works very well. I guess once the online method is ready I can really experiment with the window size. This net (gen6) was trained using the offline method again, took 2hrs 8min. Online renders my system completely useless for work and takes much longer, thats why we need V2 (:

@glinscott if you would do the honors of updating the graph :) We've passed 2K elo :collision:

[kiudee]

This network is a noticeable upgrade in strength. Here is a game with 10k playouts against Lichess Stockfish Level 1:

[Zeta36]

Who is who @kiudee? is white our Leela?

[kiudee]

@Zeta36 Sorry forgot to mention. lc_gen6 is white.

Here is another game - this time winning against Lichess Level 2 with 100k playouts and white:

Seems like it learned to value queens highly (look at how it is running away with her to safety rather than mate the king).

edit: Level 2 corresponds to

AI level 2: Stockfish v9 skill 6/20, depth 2, 100ms per move

[jkiliani]

I guess I have to conclude that the increased playouts make all the difference for Leela Chess. I just tried to match it against Stockfish Level 0 with 25 ms/move with its standard settings

./cutechess-cli -rounds 100 -tournament gauntlet -concurrency 2 -pgnout SF0.pgn \
 -engine name=lc_gen6 cmd=lczero arg="--threads=1" arg="--weights=$WDR/gen6-64x6-28000.txt" arg="--playouts=800" arg="--noponder" arg="--noise" tc=inf \
 -engine name=sf_lv0 cmd=stockfish_x86-64 option.Threads=1 option."Skill Level"=0 tc=40/1 \
 -each proto=uci

with the result:

Score of lc_gen6 vs sf_lv0: 0 - 16 - 1  [0.029] 17
Elo difference: -607.41 +/- nan

The one draw by the 50-move rule is more than I've ever seen so far in matches against a weakened Stockfish so I suppose that's progress already. I'll do a longer match for the next generation.

[glinscott]

@Error323 graph is updated! Congrats, that's a big jump in strength :). And only a few hours training!

[jkiliani]

After seeing gen6 draw once against Stockfish with skill level 0, I reconsidered and went through with a 100 game match to see how big the gap still is. Here it goes:

Score of lc_gen6 vs sf_lv0: 3 - 95 - 2  [0.040] 100
Elo difference: -552.08 +/- 257.07

I'll try this again with the next generation net, to see if the score against sf_lv0 narrows at a rate comparable to what the self-play Elo suggests.

Combined with the tests I posted in https://github.com/glinscott/leela-chess/issues/109#issuecomment-372701765, this suggests that gen6 is something like 1000 Elo weaker than the supervised net kbb1-64x6-796000.txt. I expect that the self-play Elo gain needed to beat it is probably more than that.

[Error323]

Score of lc_gen7 vs lc_gen6: 91 - 6 - 3 [0.925] 100 Elo difference: 436.43 +/- 143.57

%YAML 1.2
---
name: 'online-64x6'
gpu: 0

dataset:
    num_chunks: 100000
    train_ratio: 0.90
    input: '/mnt/storage/home/fhuizing/chess/distributed-dataset/run1/'

training:
    batch_size: 2048
    total_steps: 50000
    shuffle_size: 524288
    lr_values:
        - 0.01
        - 0.001
        - 0.0005
    lr_boundaries:
        - 35000
        - 45000
    policy_loss_weight: 1.0
    value_loss_weight: 1.0
    path: '/mnt/storage/home/fhuizing/chess/networks'

model:
    filters: 64
    residual_blocks: 6
...

Fully trained with the online pipeline V2! The yaml file shows the training parameters.

[jkiliani]

Wow that's great, have to do another round vs Stockfish now! And NN cache just got merged too so our game rate will go up a lot as well...

Current score:

Score of lc_gen7 vs sf_lv0: 3 - 16 - 0  [0.158] 19

The difference seems to be narrowing considerably, just not by 436 Elo. Let's see how the final result looks like...

[jkiliani]

I finished the match of lc_gen7 (800 playouts) against Stockfish Level 0, at 1 sec/40 moves time control. The new network cut Stockfish's margin by a lot, more than my last post suggested:

Score of lc_gen7 vs sf_lv0: 23 - 75 - 2  [0.240] 100
Elo difference: -200.24 +/- 81.11

The Elo difference decrease to https://github.com/glinscott/leela-chess/issues/100#issuecomment-372798753 does not quite match @Error323's measured 436 Elo gain, but it's not that far off. I think the next network against sf_lv0 will be really interesting. After this it may already be time to increase Stockfish's skill level...

[jjoshua2]

@Error323 does this mean we are close to being automated? Since each training has such a big gain it means it would be good to do it even every 10k games it seems.

[glinscott]

@Error323 awesome work! Graph updated on main page. I really need to finish the match support at this point. It's getting closer, but a fair bit of code, as can be seen here: https://github.com/glinscott/leela-chess/compare/match_support.

[Error323]

@jjoshua2 yes, but not quite there yet. As @glinscott said, the match support code still requires a bit of work. And I also want to build up some more confidence with the online training before letting it run without looking :see_no_evil:

Regarding the new gen, I made a mistake :sweat_smile: had to redo the training so it's gonna take a bit longer.

[Error323]

Generation 8:

Score of lc_gen8 vs lc_gen7: 82 - 14 - 4  [0.840] 100
Elo difference: 288.06 +/- 94.51
Finished match

[Zeta36]

Again a big step forward. Congrats!

[jkiliani]

Congrats on the training, but I'm afraid I have some (still tentative) bad news: I ran the new net against Stockfish Level 0, expecting it to win narrowly. To my surprise, so far:

Score of lc_gen8 vs sf_lv0: 9 - 24 - 0  [0.273] 33

Obviously the statistics here are still very weak, but it doesn't look like the self-play rating transfers to an external measurement this time. I'll run the whole 100 matches and report back then.

If the current trend is confirmed, we may be experiencing the self-play rating inflation, a phenomenon that Leela Zero experienced when validating its progress curve on http://www.yss-aya.com/cgos/19x19/standings.html. An even more extreme example was the 9x9 Go experiment by @Alderi-Tokori, who trained a long sequence of nets, each of which beat their predecessors easily, while matches against external bots confirmed that progress had actually stalled (https://github.com/gcp/leela-zero/issues/863#issuecomment-372243689).

If I can confirm these results, one thing we could try is training on a larger window. While training on a small window tends to find nets that beat the previous one more easily, it seems to also exacerbate the self-play rating inflation.

[killerducky]

LZGo has a ratio of about 3:1 self play Elo to CGOS Elo. Other Go programmers reported similar trends.

[Error323]

Currently playing against gen6 and 5 after your post and it's quite convincing so far. I've seen games where it got ahead with 13-0 and still came out equal strength, I think waiting is important :)

Score of lc_gen8 vs lc_gen6: 30 - 0 - 0  [1.000] 30
Score of lc_gen8 vs lc_gen5: 18 - 0 - 0 [1.000] 18

edit: added gen5. @jkiliani it could just have learned something smart given it's previous self which is trivial for stockfish l0. E.g. "rook is important".

[jjoshua2]

Looking at the play (too small sample size I know), I felt it was a little stronger with a lot more pawn captures, and not missing mate in 1s, and a lot of times when a piece was left hanging it was immediately captured, but still very bad since a lot free pieces were not taken. I was excited when it did a fork, but then didn't capture.

[jkiliani]

@Error323 Sounds plausible... Likely, it's a good idea to promote even if a match fails for that reason, since Deepmind adopted that procedure for Alpha zero.

[glinscott]

@Error323 great stuff! Really exciting to see the progress march forward. Now, to get the match support in... It's getting closer!

[jhellis3]

The Elo inflation is not so much a self-play issue as it is a single-opponent issue. The magnitude of the inflation will also decrease significantly over time as overall strength improves.

I am not sure promoting regardless of results is the best way to go. Another option would be to play a gauntlet vs self (current best) and the two previous networks (regardless of if they passed). This would limit the self-play games to 1/3rd, which should reduce the over-fitting/inflation issue.

[jkiliani]

I now have the result of the match against Stockfish Level 0:

Score of lc_gen8 vs sf_lv0: 34 - 65 - 1  [0.345] 100
Elo difference: -111.37 +/- 72.46

There was after all a modest gain compared to the last net, but we probably just have to expect self-play Elo to correspond to roughly 1/3 Bayes Elo with multiple, varied opponents. The type of gauntlet run by @CMCanavessi seems to be the best way to do that, I just don't have the resources for something on this scale. I'll continue the matches with handicapped Stockfish to see how the progression there corresponds to self-play Elo and to @CMCanavessi's tournament.

[jjoshua2]

[White "lczero gen8 100k"] [Black "stockfish_9 lvl 2"] 100k playouts vs skill level 2. TC set to 1 ply but it only seemed to limit SF, which was fine since it was still a close game. Tried to get chesstempo but couldn't figure out how to link it.

1. Nc3 {+0.12/24 52s} d5 {+0.72/1 0.001s} 2. a4 {+0.18/25 54s} e5 {+1.86/1 0s}
3. Rb1 {+0.17/25 60s} Nf6 {+2.64/1 0s} 4. e3 {+0.09/25 58s} c6 {+2.03/1 0.001s}
5. d3 {+0.07/25 64s} Bg4 {+2.55/1 0s} 6. f3 {+0.50/25 65s} Bc8 {+2.05/1 0s}
7. Nge2 {+0.50/25 65s} Nbd7 {+2.08/1 0s} 8. Ng3 {+0.64/25 62s} Bb4 {+2.38/1 0s}
9. Bd2 {+0.48/25 57s} h5 {+2.40/1 0s} 10. h4 {+0.58/25 67s} d4 {+2.13/1 0s}
11. exd4 {+1.45/25 67s} exd4 {+1.21/1 0.001s} 12. Nce2 {+1.46/25 65s}
Bd6 {+0.87/1 0.001s} 13. Nf5 {+1.44/25 59s} Ne5 {-1.52/1 0.001s}
14. Nxd6+ {+1.42/25 63s} Qxd6 {+0.49/1 0s} 15. b4 {+1.42/25 62s}
Bd7 {+0.85/1 0s} 16. Nxd4 {+1.63/25 64s} Nxd3+ {+3.20/1 0s}
17. Bxd3 {+2.32/25 68s} Qxd4 {-0.35/1 0.001s} 18. Qe2+ {+2.26/25 65s}
Kf8 {-0.35/1 0s} 19. c3 {+2.35/25 64s} Qd6 {+1.88/1 0s} 20. Rd1 {+2.53/25 67s}
Re8 {+4.56/1 0s} 21. Be3 {+1.38/25 68s} Qb8 {+3.81/1 0s} 22. b5 {+2.05/25 69s}
Nd5 {+3.88/1 0.001s} 23. Bc5+ {-1.32/25 66s} Kg8 {+4.07/1 0s}
24. Be4 {-1.14/25 67s} b6 {+2.30/1 0.001s} 25. Bd4 {+1.08/25 65s}
Rh6 {+0.52/1 0.001s} 26. Kf1 {+1.89/25 69s} c5 {+0.71/1 0s}
27. Bxg7 {+2.10/25 67s} Nf4 {-1.68/1 0.001s} 28. Qd2 {+1.95/25 65s}
Bxb5+ {-0.11/1 0.001s} 29. axb5 {+2.25/25 52s} Kxg7 {-0.11/1 0.001s}
30. g3 {+2.25/25 66s} Rxe4 {-0.64/1 0.001s} 31. fxe4 {+2.86/25 63s}
Ne6 {-0.86/1 0.001s} 32. Qd7 {+2.94/25 62s} Qxg3 {+2.87/1 0.001s}
33. Ke2 {+2.66/25 63s} Rg6 {+6.30/1 0.001s} 34. Rdg1 {+2.97/24 54s}
Qf4 {+2.35/1 0.001s} 35. Rxg6+ {+3.23/25 56s} Kxg6 {+0.15/1 0s}
36. Qc6 {+3.26/25 57s} Qg3 {+0.90/1 0.001s} 37. Ra1 {+5.14/24 43s}
Kg7 {+1.47/1 0.001s} 38. Rxa7 {+5.33/24 57s} Qh3 {+1.22/1 0.001s}
39. e5 {+6.57/24 58s} Qf5 {-0.38/1 0.001s} 40. Qd6 {+6.47/24 46s}
Qc2+ {-0.26/1 0.001s} 41. Kf1 {+6.43/25 50s} Qb1+ {-0.30/1 0.001s}
42. Kf2 {+7.00/24 32s} Qf5+ {-0.67/1 0.001s} 43. Ke3 {+6.70/24 49s}
Nf4 {+0.01/1 0.001s} 44. Qf6+ {+14.53/23 30s} Qxf6 {-5.50/1 0.001s}
45. exf6+ {+14.60/23 2.6s} Kh6 {-5.50/1 0s} 46. Kxf4 {+16.91/23 25s}
c4 {-6.30/1 0s} 47. Kf5 {+26.88/20 2.5s} Kh7 {-7.43/1 0s}
48. Ra8 {+34.46/18 0.52s} Kh6 {-M2/1 0s} 49. Rh8# {+39.10/16 0.29s, White mates}
1-0

[White "lczero gen8 100k"]
[Black "Ruffian_105"]
[TimeControl "1 ply/move"]
[Result "1-0"]

1.Nc3 e5 2.Nf3 Nc6 { A00: Van Geet Opening: Reversed Nimzowitsch Variation, 2.Nf3 Nc6 } 3.Rb1 { (+0.23 → -0.84) Inaccuracy. A better move was 3. e4. } ( 3.e4 3...Nf6 4.Bc4 4...Bc5 5.d3 5...O-O 6.O-O 6...h6 7.Nd5 7...a6 ) 3...d5 4.e4 Nf6 { (-1.21 → +0.25) Mistake. The best move was 4... dxe4. } ( 4...dxe4 5.Nxe4 5...f5 6.Nc3 6...e4 7.Ng1 7...Nf6 8.Bb5 8...Bc5 9.d3 ) 5.exd5 Nxd5 6.d3 { (+0.34 → -0.35) Inaccuracy. A better move was 6. Bb5. } ( 6.Bb5 6...Nxc3 7.bxc3 7...Bd6 8.O-O 8...O-O 9.Re1 9...Be6 10.d4 10...exd4 ) 6...Bc5 7.a4 O-O 8.Ne4 Bb4+ 9.c3 Ba5 10.b4 Bb6 { (-0.41 → +1.09) Mistake. The best move was 10... Nxc3. } ( 10...Nxc3 11.Nxc3 11...Nxb4 12.Bd2 12...Bf5 13.Qb3 13...Nxd3+ 14.Bxd3 14...Qxd3 15.Rb2 ) 11.b5 { (+1.12 → -0.45) Mistake. The best move was 11. a5. } ( 11.a5 11...Bxf2+ 12.Kxf2 12...f5 13.Bg5 13...Qe8 14.Qb3 14...Qf7 15.Nc5 15...b6 ) 11...Na5 12.Qc2 Re8 { (-0.49 → +0.66) Inaccuracy. A better move was 12... c5. } ( 12...c5 13.Be2 13...f5 14.Ng3 14...Qd6 15.O-O 15...Nf4 16.Re1 16...c4 17.dxc4 ) 13.Ke2 { (+0.58 → -3.68) Blunder. The best move was 13. c4. } ( 13.c4 13...c6 14.cxd5 14...cxd5 15.Ng3 15...f5 16.Be3 16...Bxe3 17.fxe3 17...f4 ) 13...Bg4 { (-3.74 → -2.62) Inaccuracy. A better move was 13... f5. } ( 13...f5 14.Ke1 14...fxe4 15.dxe4 15...Nf4 16.h4 16...Bg4 17.Ng5 17...Qf6 18.f3 ) 14.Bd2 { (-2.37 → -3.72) Inaccuracy. A better move was 14. h3. } ( 14.h3 14...Bxf3+ 15.gxf3 15...f5 16.Bg5 16...Nf4+ 17.Kd1 17...Qd5 18.c4 18...Qd4 ) 14...Qh4 { (-4.22 → -1.69) Blunder. The best move was 14... f5. } ( 14...f5 15.h3 15...Bxf3+ 16.gxf3 16...fxe4 17.dxe4 17...Qh4 18.Rh2 18...Nf4+ 19.Bxf4 ) 15.h3 Rad8 { (-2.12 → -0.06) Blunder. The best move was 15... Nf4+. } ( 15...Nf4+ 16.Ke1 16...Bxf3 17.gxf3 17...f5 18.Ng3 18...Rad8 19.Qa2+ 19...Kh8 20.Qc2 ) 16.hxg4 Qxh1 17.Ng3 { (0.00 → -2.20) Mistake. The best move was 17. c4. } ( 17.c4 17...Nf4+ 18.Bxf4 18...exf4 19.c5 19...Qh6 20.cxb6 20...cxb6 21.Ke1 21...Qg6 ) 17...Nf4+ 18.Kd1 Qh6 19.g5 Qe6 20.d4 { (-1.98 → -3.06) Inaccuracy. A better move was 20. Ne4. } ( 20.Ne4 20...c5 21.c4 21...Qd7 22.Bxf4 22...exf4 23.Kc1 23...Bc7 24.Kb2 24...b6 ) 20...Qg4 { (-3.30 → -1.56) Mistake. The best move was 20... c5. } ( 20...c5 21.Nxe5 21...cxd4 22.Bxf4 22...dxc3+ 23.Ke1 23...Bd4 24.Bd3 24...Bxe5 25.Bxe5 ) 21.Kc1 exd4 22.cxd4 Re6 23.Kb2 Red6 { (-1.93 → +0.10) Blunder. The best move was 23... g6. } ( 23...g6 24.Rc1 24...c6 25.Kb1 25...Nd5 26.Bd3 26...Re7 27.Qa2 27...Rc7 28.Be4 ) 24.Ne4 { (+0.23 → -2.16) Mistake. The best move was 24. Qf5. } ( 24.Qf5 24...Qxf5 25.Nxf5 25...Ne6 26.Nxd6 26...Rxd6 27.Re1 27...Rd8 28.Bc3 28...c5 ) 24...Rd5 25.Bc3 Nxg2 26.Ne5 Qh3 { (-1.75 → -1.04) Inaccuracy. A better move was 26... Rxe5. } ( 26...Rxe5 27.dxe5 27...Nf4 28.Re1 28...Qf3 29.Be2 29...Nxe2 30.Qxe2 30...Qf5 31.Rd1 ) 27.Ng3 { (-1.39 → -4.91) Mistake. The best move was 27. g6. } ( 27.g6 27...hxg6 28.Ng5 28...Qh4 29.Nexf7 29...Rf5 30.Nxd8 30...Rxf2 31.Be2 31...Nf4 ) 27...Bxd4 28.Bxd4 Rxd4 29.Qxc7 Rb4+ { (-5.15 → -0.24) Blunder. The best move was 29... Qe6. } ( 29...Qe6 30.Qxf7+ 30...Qxf7 31.Nxf7 31...Kxf7 32.Bxg2 32...Rb4+ 33.Ka1 33...Rxa4+ 34.Kb2 ) 30.Ka3 Rxb1 { (+0.77 → ♔ Mate in 1) Blunder. The best move was 30... Rbd4. } ( 30...Rbd4 31.Qxf7+ 31...Kh8 32.Bxg2 32...Qh4 33.Be4 33...Rxe4 34.Nxe4 34...Qxe4 35.Qc7 ) 31.Qxd8#  1-0```

[Uriopass]

Here is the gif of the game for thoses who don't have the patience.

[Zeta36]

Wonderful game of Leela. There is no doubt the model is learning chess strategies.

[jjoshua2]

Here is 100k playouts against full sf9, but still limited to 1 ply. @Uriopass Note there was a game against Ruffian in the last post as well. SF won all 3 in a row no cherry picking data. I guess it's time to try ply=2 [White "lczero gen8 100k"] [Black "stockfish_9 ply 1"] [Result "1-0"]

1. Nc3 {+0.12/24 50s} d5 {+0.72/1 0.001s} 2. a4 {+0.18/25 55s} e5 {+1.86/1 0s}
3. Rb1 {+0.17/25 58s} d4 {+2.64/1 0s} 4. Nf3 {-0.05/25 61s} dxc3 {+2.89/1 0s}
5. Nxe5 {-0.68/25 64s} Qd4 {+4.16/1 0s} 6. Nf3 {-0.36/25 67s}
cxd2+ {+2.84/1 0.001s} 7. Bxd2 {-0.10/25 62s} Qxa4 {+3.23/1 0s}
8. b3 {-0.12/25 62s} Qe4 {+3.61/1 0s} 9. e3 {-0.06/25 64s} Nf6 {+3.44/1 0s}
10. Ng5 {-0.42/25 55s} Qe5 {+3.72/1 0s} 11. h4 {-0.57/25 67s} h6 {+4.28/1 0s}
12. f4 {-1.05/25 64s} Qc5 {+4.77/1 0s} 13. b4 {-0.88/25 66s} Qd5 {+4.75/1 0s}
14. Nf3 {-0.50/25 66s} Ne4 {+4.75/1 0s} 15. c3 {-0.69/25 62s} Bg4 {+5.85/1 0s}
16. Qc2 {-1.41/25 64s} Be7 {+6.00/1 0.001s} 17. c4 {+0.10/25 65s}
Qe6 {+5.24/1 0.001s} 18. Nd4 {+0.19/25 56s} Qg6 {+4.55/1 0s}
19. Bd3 {-0.27/25 65s} f5 {+3.14/1 0.001s} 20. b5 {-0.15/25 63s}
Bh5 {+5.91/1 0s} 21. Nxf5 {+0.45/25 65s} Nc5 {+2.50/1 0.001s}
22. Be4 {+0.48/25 64s} Qg4 {+4.62/1 0s} 23. Bf3 {+3.49/25 63s}
Bxh4+ {-3.86/1 0s} 24. Rxh4 {+6.38/25 52s} Qxh4+ {-6.41/1 0.001s}
25. Nxh4 {+6.49/25 57s} Bf7 {-6.50/1 0s} 26. f5 {+6.73/25 56s} Nbd7 {-6.07/1 0s}
27. Ng6 {+6.62/25 65s} Bxg6 {-6.12/1 0s} 28. fxg6 {+8.31/25 60s}
Ne5 {-5.55/1 0s} 29. Bxb7 {+8.62/25 61s} Nxb7 {-3.42/1 0s}
30. Qe4 {+9.24/25 51s} O-O-O {-6.78/1 0s} 31. Qxe5 {+9.39/25 156s}
Nd6 {-6.67/1 0s} 32. Ba5 {+9.30/25 173s} Rd7 {-6.56/1 0s}
33. c5 {+10.15/25 189s} Re8 {-6.65/1 0s} 34. Qh5 {+10.21/25 177s}
Ne4 {-6.41/1 0.001s} 35. Bb4 {+10.18/25 171s} Kb8 {-6.50/1 0s}
36. Qf5 {+10.01/25 166s} Ree7 {-6.57/1 0s} 37. c6 {+15.50/24 160s}
Nd6 {-9.90/1 0.001s} 38. Qxd7 {+16.31/24 117s} Rxd7 {-9.52/1 0s}
39. cxd7 {+16.47/24 150s} Nb7 {-6.72/1 0s} 40. Be7 {+16.54/24 158s}
c6 {-8.68/1 0s} 41. bxc6 {+21.40/24 117s} Kc7 {-14.19/1 0s}
42. cxb7 {+21.52/24 123s} Kxd7 {-16.69/1 0.001s} 43. b8=Q {+22.07/24 47s}
Kxe7 {-16.69/1 0.001s} 44. Qe5+ {+23.57/24 76s} Kd7 {-17.42/1 0.001s}
45. Qxg7+ {+24.40/24 43s} Kc8 {-17.56/1 0s} 46. Qxa7 {+33.28/22 29s}
h5 {-M2/1 0s} 47. Rb8# {+38.97/20 6.6s, White mates} 1-0

[Uriopass]

http://www.apronus.com/chess/puzzle/editor.php

[Error323]

Sorry people, tomorrow there will be a new version. I'm tracking a bug.

Which I introduced myself :cry:

[CMCanavessi]

No hurry, better slow but solid.

[jjoshua2]

gen8-100k v sf9-2ply. SF won with kind of a long queen and rook checkmate in the corner.

[Error323]

Score of lc_gen9 vs lc_gen8: 61 - 34 - 5  [0.635] 100
Elo difference: 96.19 +/- 69.82

Still an improvement, but not as big as I had hoped. Slowdown of training caused by #119. I uploaded the net from step 48'000.

[jjoshua2]

@Error323 Are you saying that more training would likely have made a stronger net? Or just there was a long time between gen 8 and gen 9. Hopefully gen 10 will be less of a problem and also stronger!

Played gen 9 100k against Ruffian ply 2 and it lost fast 0-1. I'm wondering if Na3 is common because bishop takes and pawn retakes, like in this game, and bishops are worth a little more?

1. Na3 {0.00/25 163s} e5 {-0.27/2 0.001s} 2. h4 {+0.02/25 181s}
Bxa3 {+0.03/2 0.001s} 3. bxa3 {+0.25/25 170s} d5 {-0.17/2 0s}
4. Rb1 {+0.17/25 168s} Nf6 {-0.08/2 0.001s} 5. d3 {+0.17/25 53s}
Nc6 {-0.01/2 0.001s} 6. e3 {+0.35/25 61s} O-O {+0.16/2 0.010s}
7. Ne2 {+0.61/25 59s} Qd6 {+0.21/2 0s} 8. Rg1 {+1.03/25 59s}
Ng4 {+0.31/2 0.001s} 9. d4 {+1.29/25 65s} Qf6 {+2.01/2 0.002s}
10. f3 {+1.20/25 65s} Nh2 {+0.56/2 0.002s} 11. c4 {+1.07/25 63s}
Qxh4+ {+3.05/2 0.010s} 12. Kd2 {+0.65/25 69s} exd4 {+3.04/2 0.006s}
13. cxd5 {+1.04/25 66s} Nxf1+ {+3.23/2 0.004s} 14. Rxf1 {+0.75/25 68s}
dxe3+ {+3.28/2 0.001s} 15. Kxe3 {+0.36/25 68s} Re8+ {+4.04/2 0.001s}
16. Kd2 {0.00/25 67s} Nd4 {+4.16/2 0.002s} 17. Ng1 {+0.08/25 66s}
Bf5 {+3.47/2 0.002s} 18. Rb2 {-1.57/25 66s} Rad8 {+3.96/2 0.002s}
19. Kc3 {-2.11/25 59s} Rxd5 {+3.78/2 0.002s} 20. Rxb7 {-2.49/25 60s}
Nb5+ {+5.49/2 0.003s} 21. Kb3 {-6.00/25 58s} Rd3+ {+6.76/2 0.003s}
22. Qxd3 {-7.69/25 60s} Bxd3 {+6.54/2 0.002s} 23. Rd1 {-7.64/25 58s}
Qc4+ {+M3/1 0s} 24. Kb2 {-38.67/17 1.1s} Qc3# {+M1/1 0s, Black mates} 0-1

[Error323]

@Error323 Are you saying that more training would likely have made a stronger net? Or just there was a long time between gen 8 and gen 9. Hopefully gen 10 will be less of a problem and also stronger!

Just a long time between. I'm also running experiments to find optimizations, I may have some interesting results. But the downside is it'll take longer for new nets to arrive. Once #119 is merged, I can do both at the same time again. And once match evaluation is done by @glinscott we can iterate faster.

[Error323]

That said, is there anyone around here that can help me on the Windows side with #119? Could one of you please get the AppVeyor build https://ci.appveyor.com/project/glinscott/leela-chess/build/90. And run the client with the gen8 network (1aa02cb1).

./lczero --weights=1aa02cb1 --seed=1 --start='train 1 1' -t1 -q

And once done, upload the training.0.gz file in the data-1 directory here.

[CMCanavessi]

I can do that in 20 minutes when a 40/40 game is finished.

[jjoshua2]

I'll try. DL the file https://ci.appveyor.com/api/buildjobs/nxu1vkp3gyhwxol9/artifacts/build%2FRelease%2Flczero.exe

[Error323]

It's important to get the exe from that specific branch #119 and run it with the exact arguments shown:

./lczero --weights=1aa02cb1 --seed=1 --start='train 1 1' -t1 -q

Also make sure that before you start the data-1 directory is empty or non existent.

[jjoshua2]

I got that link from your link and clicking the top one that's not CPU only. It's ran. training.0.gz

[CMCanavessi]

I get the following:

F:\Chess\LCZero\test>lczero.exe --weights=1aa02cb1 --seed=1 --start='train 1 1' -t1 -q
Unrecognized argument: 1
Unrecognized argument: 1'
LCZero Copyright (C) 2017  Gary Linscott
Based on:Leela Chess Copyright (C) 2017 benediamond
Leela Zero Copyright (C) 2017  Gian-Carlo Pascutto
Stockfish Copyright (C) 2017  Tord Romstad, Marco Costalba, Joona Kiiski, Gary Linscott
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; see the COPYING file for details.

Allowed options:
  -h [ --help ]                Show commandline options.
  -t [ --threads ] arg (=2)    Number of threads to use.
  -p [ --playouts ] arg        Weaken engine by limiting the number of
                               playouts. Requires --noponder.
  -r [ --resignpct ] arg (=10) Resign when winrate is less than x%.
  -n [ --noise ]               Apply dirichlet noise to root.
  --randomize                  Randomize move selection at root (only useful
                               for training).
  -s [ --seed ] arg            Random number generation seed.
  -w [ --weights ] arg         File with network weights.
  -l [ --logfile ] arg         File to log input/output to.
  -q [ --quiet ]               Disable all diagnostic output.
  --noponder                   Disable thinking on opponent's time.
  --start arg                  Start command {train, bench}.
  --supervise arg              Dump supervised learning data from the pgn.
  --gpu arg                    ID of the OpenCL device(s) to use (disables
                               autodetection).
  --full-tuner                 Try harder to find an optimal OpenCL tuning.
  --tune-only                  Tune OpenCL only and then exit.

F:\Chess\LCZero\test>

[CMCanavessi]

Changed the single quotes to double quotes and looks to be working

[Error323]

Ah excellent :)

[jjoshua2]

I used powershell instead of command prompt and didn't have issues.

[Error323]

@jjoshua2 could you post the PGN?

[CMCanavessi]

training.0.gz

Here's the generated file

[Error323]

Could you also post the PGN? :)

[jjoshua2]

@Error323 I uploaded the gz two posts ago. I don't see a pgn, just a training.0 instead of a gz. Edit I guess its in my window still...

[CMCanavessi]

Full output, with PGN at the bottom:

F:\Chess\LCZero\test>lczero.exe --weights=1aa02cb1 --seed=1 --start="train 1 1" -t1 -q
Created dirs data-1
info depth 16 nodes 800 nps 677 score cp 15 winrate 50.61% time 1170 pv d2d3 g8f6 g1f3 d7d6 e2e4
info depth 16 nodes 800 nps 800 score cp -3 winrate 49.86% time 990 pv a7a5 g1f3 g8f6 b1c3 f6g4
info depth 16 nodes 800 nps 769 score cp -6 winrate 49.76% time 1030 pv g1f3 g8f6 b1a3 f6g4
info depth 16 nodes 800 nps 769 score cp -11 winrate 49.56% time 1030 pv g8f6 a2a4 d7d6 b1c3
info depth 17 nodes 800 nps 784 score cp -13 winrate 49.47% time 1010 pv b1c3 d7d6 a2a4 f6g4
info depth 17 nodes 800 nps 769 score cp 4 winrate 50.18% time 1030 pv f6g4 e2e4 g4h2 f3h2 d7d6
info depth 17 nodes 800 nps 769 score cp 22 winrate 50.88% time 1030 pv h2h3 d7d6 h3g4 c8g4 h1g1 g4f3 e2f3
info depth 17 nodes 800 nps 800 score cp -28 winrate 48.91% time 990 pv g4f6 a2a4 b7b6 f3g5
info depth 17 nodes 800 nps 761 score cp 6 winrate 50.25% time 1040 pv a2a4 b8c6 c3b5 c6b4
info depth 17 nodes 800 nps 800 score cp -29 winrate 48.87% time 990 pv b8c6 e2e4 c6b4 e4e5
info depth 17 nodes 800 nps 800 score cp 10 winrate 50.41% time 990 pv a1b1 c6b4 e2e4 f6e4 d3e4
info depth 16 nodes 800 nps 754 score cp -55 winrate 47.86% time 1050 pv d7d5 c1d2 d5d4 c3b5
info depth 17 nodes 800 nps 754 score cp 62 winrate 52.39% time 1050 pv c1g5 h7h6 g5f6 g7f6 e2e4
info depth 17 nodes 800 nps 769 score cp -39 winrate 48.48% time 1030 pv h7h6 g5f4 d5d4 c3b5 c6b4
info depth 17 nodes 800 nps 800 score cp -1 winrate 49.93% time 990 pv g5h4 c8d7 e2e3 g7g5 h4g5
info depth 17 nodes 800 nps 909 score cp -23 winrate 49.10% time 870 pv d5d4 e2e4 d4c3 b2c3 b7b6 e4e5
info depth 17 nodes 800 nps 824 score cp 0 winrate 50.04% time 960 pv e2e4 d4c3 b2c3 b7b6 e4e5 c8b7
info depth 17 nodes 800 nps 824 score cp -7 winrate 49.72% time 960 pv d4c3 b2c3 c8h3 g2h3 f6e4 d3e4 e7e6
info depth 17 nodes 800 nps 952 score cp -23 winrate 49.09% time 830 pv b2c3 g7g5 f3g5 h6g5 h4g5 h8g8 g5f6 e7f6
info depth 17 nodes 800 nps 4210 score cp -2 winrate 49.89% time 180 pv e7e5 h4g3 h6h5 f3e5
info depth 17 nodes 800 nps 776 score cp -22 winrate 49.15% time 1020 pv g2g4 g7g5 h4g5 h6g5 f3g5 c8g4 h3g4
info depth 17 nodes 800 nps 792 score cp 14 winrate 50.56% time 1000 pv g7g5 f3g5 h6g5 h4g5 c8g4 g5f6 g4d1 f6d8
info depth 17 nodes 800 nps 833 score cp -125 winrate 45.20% time 950 pv h4g3 c8d7 f3e5 c6e5 g3e5 d7a4 e5f6 d8f6
info depth 17 nodes 800 nps 1025 score cp 79 winrate 53.05% time 770 pv f6g4 h3g4 c8g4 g3e5 c6e5 f3e5
info depth 17 nodes 800 nps 833 score cp -78 winrate 46.97% time 950 pv h3g4 c8g4 h1g1 g4f3 d1f3 g5g4 f3g4 f8e7
info depth 17 nodes 800 nps 987 score cp 107 winrate 54.13% time 800 pv c8g4 b1b7 g4f3 d1f3 f8e7 f3d1 g5g4 g3e5
info depth 17 nodes 800 nps 2758 score cp -155 winrate 44.06% time 280 pv f1e2 h8g8 f3e5 c6e5 g3e5 g4e2
info depth 17 nodes 800 nps 851 score cp 98 winrate 53.78% time 930 pv f7f6 f3e5 c6e5 e2g4 e5g4 d1g4 c7c6
info depth 17 nodes 800 nps 869 score cp -44 winrate 48.29% time 910 pv f3g5 g4e2 d1e2 f6g5 e2h5 e8e7 g3e5 c6e5
info depth 17 nodes 800 nps 1379 score cp 49 winrate 51.89% time 570 pv g4e2 d1e2 f6g5 f2f3 h8g8 g3e5 c6e5 b1d1
info depth 17 nodes 800 nps 2285 score cp -130 winrate 45.01% time 340 pv d1e2 f6g5 h1g1 h8g8 g3e5 c6e5 b1d1 f8c5
info depth 17 nodes 800 nps 2162 score cp 89 winrate 53.44% time 360 pv f6g5 f2f3 h8g8 b1b7 f8e7 g3e5
info depth 17 nodes 800 nps 2285 score cp -154 winrate 44.08% time 340 pv f2f3 h8g8 b1b7 f8e7 g3e5 c6e5
info depth 17 nodes 800 nps 792 score cp 155 winrate 55.94% time 1000 pv h6h5 b1b2 h8g8 g3e5 c6e5
info depth 17 nodes 800 nps 808 score cp -148 winrate 44.32% time 980 pv e2g2 h5h4 g3h4 g5h4 b1d1 h4h3 g2h3
info depth 17 nodes 800 nps 833 score cp 120 winrate 54.60% time 950 pv d8b8 h1g1 g5g4 f3g4
info depth 17 nodes 800 nps 754 score cp -103 winrate 46.05% time 1050 pv e1e2 f8e7 g3e5 c6e5 d3d4
info depth 17 nodes 800 nps 761 score cp 184 winrate 57.03% time 1040 pv h5h4 e2e3 h4h3 g2h3 h8h3 h1h3
info depth 17 nodes 800 nps 879 score cp -155 winrate 44.06% time 900 pv g3e1 f8h6 e1h4 g5h4 h1h4 h8g8 h4h6
info depth 17 nodes 800 nps 963 score cp 117 winrate 54.52% time 820 pv f8h6 e1h4 g5h4 h1h4 h6g7 h4h1 h8h1
info depth 17 nodes 800 nps 941 score cp -134 winrate 44.83% time 840 pv e1h4 g5h4 h1h4 h8g8 h4h6 g8g2 e2e3
info depth 17 nodes 800 nps 800 score cp 169 winrate 56.50% time 990 pv g5h4 h1h4 b8a7 h4h6 h8h6 d3d4 e5d4 c3d4
info depth 17 nodes 800 nps 4000 score cp -208 winrate 42.07% time 190 pv h1h4 b8a7 h4h6 h8h6 d3d4 e5d4 c3d4
info depth 17 nodes 800 nps 1081 score cp 189 winrate 57.24% time 730 pv h6g7 h4h8 g7h8 g2g6 e8e7 b1b2 c6d8
info depth 17 nodes 800 nps 1095 score cp -192 winrate 42.65% time 720 pv h4g4 g7f6 g2g3 h8g8 g4g8 e8d7
info depth 17 nodes 800 nps 1066 score cp 122 winrate 54.69% time 740 pv g7f6 g2g1 b8d8 g1g3 h8f8 g3h4
info depth 17 nodes 800 nps 1481 score cp -188 winrate 42.79% time 530 pv g2g1 b8d8 g1g3 f6e7 g4g7
info depth 17 nodes 800 nps 824 score cp 230 winrate 58.78% time 960 pv c6e7 g4g2 c7c6 b1b2 h8f8 b2b1
info depth 17 nodes 800 nps 1269 score cp -293 winrate 38.91% time 620 pv g4g2 c7c6 b1b2 h8f8 b2b1
info depth 17 nodes 800 nps 1142 score cp 264 winrate 60.04% time 690 pv h8h3 b1b2 c7c6 g1f2 b8c7 f2g1
info depth 17 nodes 800 nps 1012 score cp -301 winrate 38.59% time 780 pv b1f1 c7c6 g1f2 b8c7 f2g1 h3h4
info depth 17 nodes 800 nps 824 score cp 239 winrate 59.11% time 960 pv c7c6 g1f2 b8c7 f2g1 h3h4 g2g3
info depth 17 nodes 800 nps 879 score cp -308 winrate 38.37% time 900 pv e2e1 h3h4 e1f2 h4h2 g1h2
info depth 17 nodes 800 nps 919 score cp 326 winrate 62.29% time 860 pv b8a7 e1e2 a7g1 f1g1 h3h4 g2g3 h4h2 e2d1 f6h4
info depth 16 nodes 800 nps 1600 score cp -388 winrate 35.47% time 490 pv e1d1 a7g1 g2g1 h3h2 d3d4 e5d4 c3d4
info depth 16 nodes 800 nps 1000 score cp 440 winrate 66.32% time 790 pv a7g1 g2g1 h3h2 c3c4 a8b8 d3d4 e5d4
info depth 16 nodes 800 nps 2758 score cp -538 winrate 30.40% time 280 pv g2g1 h3h2 d1e1 e8f8 d3d4 e5d4 c3d4
info depth 16 nodes 800 nps 1777 score cp 519 winrate 69.00% time 440 pv h3h2 c3c4 f6h4 g1g2 h2g2 c2c3 g2h2
info depth 16 nodes 800 nps 1111 score cp -545 winrate 30.18% time 710 pv c3c4 e8f8 c4c5 h2h4 d3d4 e5d4
info depth 16 nodes 800 nps 888 score cp 559 winrate 70.29% time 890 pv e8f7 c2c3 f6h4 g1g2 h2g2 d3d4 e5d4
info depth 16 nodes 800 nps 888 score cp -547 winrate 30.12% time 890 pv c4c5 f6h4 g1h1 h2h1 f1h1 h4f6 h1g1 f7e6 d1e2 e7c8
info depth 16 nodes 800 nps 987 score cp 511 winrate 68.73% time 800 pv f6g7 c2c3 g7f6 c3c4 f6h4 g1g2
info depth 16 nodes 800 nps 869 score cp -491 winrate 31.95% time 910 pv f3f4 e5f4 f1f4 f7g8 f4f3 g8h7 g1g2 h2g2
info depth 16 nodes 800 nps 888 score cp 374 winrate 64.01% time 890 pv e5f4 f1f4 f7g8 f4f3 g8h7 g1g7 h7g7 f3f7
info depth 16 nodes 800 nps 1290 score cp -398 winrate 35.13% time 610 pv f1f4 f7g8 f4g4 h2h7 d1e2 g8f7 g4g7 f7f8 g7h7
info depth 16 nodes 800 nps 4000 score cp 340 winrate 62.80% time 190 pv f7g8 f4g4 h2h7 g4g2 h7h5 g2g4 h5h2 g4g7
info depth 16 nodes 800 nps 3076 score cp -380 winrate 35.76% time 250 pv f4g4 h2h7 g4g2 h7h5 g2g4 h5h2 g4g7 g8f8
info depth 16 nodes 800 nps 1403 score cp 348 winrate 63.10% time 560 pv h2h7 g4g2 a8e8 d1c1 e7f5 e4f5
info depth 16 nodes 800 nps 3333 score cp -428 winrate 34.10% time 230 pv g4g2 a8e8 d1c1 e7f5 e4f5
info depth 16 nodes 800 nps 1111 score cp 415 winrate 65.45% time 710 pv g8h8 d1e2 g7c3 g1f1 c3b2
info depth 16 nodes 800 nps 1311 score cp -491 winrate 31.96% time 600 pv g1f1 g7c3 f1f3 c3b2 f3f1 b2c3
info depth 16 nodes 800 nps 1000 score cp 425 winrate 65.79% time 790 pv e7g8 f1f3 g7a1 f3f5 h7g7
info depth 16 nodes 800 nps 898 score cp -503 winrate 31.56% time 880 pv f1f3 g7f8 f3f5 f8g7 g2g7 h7g7
info depth 16 nodes 800 nps 860 score cp 533 winrate 69.45% time 920 pv a8f8 f3g3 g7c3 g3f3 f8f3
info depth 16 nodes 800 nps 1159 score cp -602 winrate 28.36% time 680 pv d1e2 f8f3 e2f3 g7b2 g2g8 h8g8 f3e3 g8f7 d3d4
info depth 16 nodes 800 nps 1025 score cp 545 winrate 69.82% time 770 pv f8f3 e2f3 g7d4 g2g5 d4c5 g5c5 b7b6 c5c6 g8e7
info depth 16 nodes 800 nps 2500 score cp -615 winrate 27.95% time 310 pv e2f3 g7f8 f3f2 f8c5 f2e2 c5b6 g2g8 h8g8 e4e5
info depth 16 nodes 800 nps 13333 score cp 600 winrate 71.58% time 50 pv g7f8 f3f2 f8c5 f2e2 c5b6 g2g8 h8g8 e4e5
info depth 16 nodes 800 nps 1066 score cp -600 winrate 28.41% time 740 pv d3d4 g8f6 e4e5 f6d5 f3e4 d5b4 e5e6 b4c2
info depth 16 nodes 800 nps 1176 score cp 519 winrate 68.98% time 670 pv f8e7 g2d2 h7h5 f3f4 e7f6
info depth 16 nodes 800 nps 842 score cp -488 winrate 32.05% time 940 pv g2d2 g8h6 d4d5 e7c5 d5c6 b7c6 c2c3
info depth 16 nodes 800 nps 1025 score cp 447 winrate 66.58% time 770 pv e7g5 d2d1 g5f6 c2c3 f6d4 c3d4
info depth 16 nodes 800 nps 941 score cp -486 winrate 32.10% time 840 pv d2d1 g5e7 d1b1 e7c5 d4c5 h7h5
info depth 16 nodes 800 nps 1632 score cp 394 winrate 64.74% time 480 pv g5f6 e4e5 f6e5 d4e5 g8e7 d1d6 e7f5
info depth 16 nodes 800 nps 869 score cp -366 winrate 36.26% time 910 pv e4e5 f6e7 d1b1 h7h3 f3g4 h3h2
info depth 16 nodes 800 nps 1481 score cp 290 winrate 60.98% time 530 pv f6e7 d1b1 h7h3 f3g2 h3h5 b1b7
info depth 16 nodes 800 nps 1025 score cp -290 winrate 38.99% time 770 pv d1b1 e7d8 e5e6 g8e7 b1b7 e7c8 b7h7 h8h7
info depth 16 nodes 800 nps 1739 score cp 232 winrate 58.84% time 450 pv e7d8 e5e6 h7h3 f3g4 h3h2 b1b7
info depth 16 nodes 800 nps 952 score cp -223 winrate 41.47% time 830 pv f3e4 d8e7 b1b7 e7c5 d4c5 h7b7 e5e6 b7e7
info depth 16 nodes 800 nps 941 score cp 160 winrate 56.13% time 840 pv d8e7 e4f5 e7d8 e5e6 g8e7 f5e5 e7d5 b1b7
info depth 16 nodes 800 nps 851 score cp 82 winrate 53.16% time 930 pv b1b7 h7h4 e4e3 h4h2 b7e7 g8e7 e5e6 e7d5
info depth 16 nodes 800 nps 2962 score cp -148 winrate 44.30% time 260 pv h7h4 e4e3 h4h2 b7e7 g8e7 e5e6 e7d5
info depth 16 nodes 800 nps 1269 score cp 106 winrate 54.10% time 620 pv e4d3 h4h3 d3c4 e7g5 e5e6 g5f6 d4d5
info depth 16 nodes 800 nps 1818 score cp -181 winrate 43.05% time 430 pv h4h3 d3c4 e7g5 e5e6 g5d8 d4d5 c6d5
info depth 16 nodes 800 nps 1666 score cp 143 winrate 55.49% time 470 pv d3c4 e7d8 b7b8 h3h6 b8d8 h8h7
info depth 16 nodes 800 nps 1600 score cp -219 winrate 41.65% time 490 pv h3e3 b7b8 h8g7 b8a8 e7d8 a8d8
info depth 16 nodes 800 nps 1012 score cp 278 winrate 60.57% time 780 pv b7c7 e7d8 c7c6 e3h3 c6d6 h3h1 d6d8
info depth 16 nodes 800 nps 1333 score cp -362 winrate 36.43% time 590 pv e7d8 c7c6 g8e7 c6d6 h8g7 d6d8 g7f7 d8d6
info depth 16 nodes 800 nps 1600 score cp 357 winrate 63.43% time 490 pv c7c6 e3e1 d4d5 e1e5 d5d6 e5c5 c4c5 h8h7
info depth 16 nodes 800 nps 6666 score cp -410 winrate 34.72% time 110 pv e3e1 d4d5 e1e5 d5d6 e5c5 c4c5 h8h7
info depth 16 nodes 800 nps 963 score cp 366 winrate 63.73% time 820 pv c4b5 h8h7 c6d6 d8c7 e5e6 c7d6
info depth 16 nodes 800 nps 1159 score cp -467 winrate 32.77% time 680 pv g8e7 c6d6 d8c7 d6d7 c7b8 d7e7 h8g8
info depth 16 nodes 800 nps 888 score cp 597 winrate 71.50% time 890 pv c6d6 d8c7 d6d7 c7e5 d4e5 e1e4 d7e7 e4a4
info depth 16 nodes 800 nps 4444 score cp -661 winrate 26.56% time 170 pv d8c7 d6d7 e1b1 b5c4 c7b8 d7e7 h8g8 c5c6
info depth 16 nodes 800 nps 1509 score cp 648 winrate 73.05% time 520 pv d6d7 e1b1 b5c4 b1b4 c4c3 b4a4 d7c7 a4a3 c3b2
info depth 16 nodes 800 nps 2051 score cp -763 winrate 23.61% time 380 pv e1b1 b5c4 b1b4 c4c3 b4a4 d7c7 a4a3 c3b2
info depth 16 nodes 800 nps 1012 score cp 716 winrate 75.07% time 780 pv b5c4 b1c1 c4b5 c1c2 d7e7 h8g8 e7c7 g8f8 b5a5
info depth 16 nodes 800 nps 1818 score cp -705 winrate 25.25% time 430 pv b1c1 c4b5 e7c8 e5e6 c1c2 e6e7 c8e7 d7e7 h8g8
info depth 15 nodes 800 nps 1012 score cp 894 winrate 79.85% time 780 pv d7c7 c1c2 c4b5 e7d5 c7a7 d5c3 b5a5
info depth 15 nodes 800 nps 2352 score cp -1062 winrate 16.32% time 330 pv c1c2 c4b5 e7d5 c7a7 d5c3 b5a5
info depth 15 nodes 800 nps 1702 score cp 1031 winrate 83.01% time 460 pv c4d3 c2a2 c7e7 a2a4 d4d5 h8g8 d5d6 g8f8
info depth 15 nodes 800 nps 1290 score cp -1098 winrate 15.58% time 610 pv c2a2 c7e7 a2a4 c5c6 h8g8 c6c7 a4d4 d3d4
info depth 15 nodes 800 nps 1290 score cp 1033 winrate 83.07% time 610 pv c7e7 a2a4 c5c6 a4a3 d3c4 a5a4 c6c7 a3a2
info depth 15 nodes 800 nps 2962 score cp -1138 winrate 14.78% time 260 pv a2a4 c5c6 h8g8 c6c7 g8f8 c7c8q f8e7 c8b7 e7e6
info depth 15 nodes 800 nps 1403 score cp 1189 winrate 86.17% time 560 pv c5c6 h8g8 c6c7 g8f8 c7c8q f8e7 c8b7 e7e6 d4d5
info depth 15 nodes 800 nps 1860 score cp -1300 winrate 11.91% time 420 pv h8g8 c6c7 g8f8 c7c8q f8e7 c8c7 e7e6 d3c3 e6d5
info depth 15 nodes 800 nps 1538 score cp 1403 winrate 89.66% time 510 pv c6c7 g8f8 e7d7 a4a3 d3c4 a3a2 c7c8q
info depth 15 nodes 800 nps 3809 score cp -1538 winrate  8.58% time 200 pv a4a3 d3c4 g8f8 e7d7 f8e8 c7c8q
info depth 15 nodes 800 nps 2352 score cp 1535 winrate 91.39% time 330 pv d3c4 g8f8 e7d7 f8e8 c7c8q
info depth 14 nodes 800 nps 5333 score cp -1710 winrate  6.72% time 140 pv g8f8 e7d7 f8e8 c7c8q
info depth 14 nodes 800 nps 4705 score cp 1757 winrate 93.73% time 160 pv e7d7 f8e8 c7c8q
info depth 13 nodes 800 nps 13333 score cp -2216 winrate  3.20% time 50 pv f8e8 c7c8q
info depth 11 nodes 800 nps 10000 score cp 2597 winrate 98.19% time 70 pv c7c8q
PGN
1. d3 a5 2. Nf3 Nf6 3. Nc3 Ng4 4. h3 Nf6 5. a4 Nc6 6. Rb1 d5 7. Bg5 h6 8.
Bh4 d4 9. e4 dxc3 10. bxc3 e5 11. g4 g5 12. Bg3 Nxg4 13. hxg4 Bxg4 14. Be2
f6 15. Nxg5 Bxe2 16. Qxe2 fxg5 17. f3 h5 18. Qg2 Qb8 19. Ke2 h4 20. Be1 Bh6
21. Bxh4 gxh4 22. Rxh4 Bg7 23. Rg4 Bf6 24. Qg1 Ne7 25. Rg2 Rh3 26. Rf1 c6
27. Ke1 Qa7 28. Kd1 Qxg1 29. Rgxg1 Rh2 30. c4 Kf7 31. c5 Bg7 32. f4 exf4
33. Rxf4+ Kg8 34. Rfg4 Rh7 35. R4g2 Kh8 36. Rf1 Ng8 37. Rf3 Rf8 38. Ke2
Rxf3 39. Kxf3 Bf8 40. d4 Be7 41. Rd2 Bg5 42. Rd1 Bf6 43. e5 Be7 44. Rb1 Bd8
45. Ke4 Be7 46. Rxb7 Rh4+ 47. Kd3 Rh3+ 48. Kc4 Re3 49. Rc7 Bd8 50. Rxc6 Re1
51. Kb5 Ne7 52. Rd6 Bc7 53. Rd7 Rb1+ 54. Kc4 Rc1 55. Rxc7 Rxc2+ 56. Kd3 Ra2
57. Rxe7 Rxa4 58. c6 Kg8 59. c7 Ra3+ 60. Kc4 Kf8 61. Rd7 Ke8 62. c8=Q+
END
Score: 1

F:\Chess\LCZero\test>

[jjoshua2]

PGN
1. d3 a5 2. Nf3 Nf6 3. Nc3 Ng4 4. h3 Nf6 5. a4 Nc6 6. Rb1 d5 7. Bg5 h6 8.
Bh4 d4 9. e4 dxc3 10. bxc3 e5 11. g4 g5 12. Bg3 Nxg4 13. hxg4 Bxg4 14. Be2
f6 15. Nxg5 Bxe2 16. Qxe2 fxg5 17. f3 h5 18. Qg2 Qb8 19. Ke2 h4 20. Be1 Bh6
21. Bxh4 gxh4 22. Rxh4 Bg7 23. Rg4 Bf6 24. Qg1 Ne7 25. Rg2 Rh3 26. Rf1 c6
27. Ke1 Qa7 28. Kd1 Qxg1 29. Rgxg1 Rh2 30. c4 Kf7 31. c5 Bg7 32. f4 exf4
33. Rxf4+ Kg8 34. Rfg4 Rh7 35. R4g2 Kh8 36. Rf1 Ng8 37. Rf3 Rf8 38. Ke2
Rxf3 39. Kxf3 Bf8 40. d4 Be7 41. Rd2 Bg5 42. Rd1 Bf6 43. e5 Be7 44. Rb1 Bd8
45. Ke4 Be7 46. Rxb7 Rh4+ 47. Kd3 Rh3+ 48. Kc4 Re3 49. Rc7 Bd8 50. Rxc6 Re1
51. Kb5 Ne7 52. Rd6 Bc7 53. Rd7 Rb1+ 54. Kc4 Rc1 55. Rxc7 Rxc2+ 56. Kd3 Ra2
57. Rxe7 Rxa4 58. c6 Kg8 59. c7 Ra3+ 60. Kc4 Kf8 61. Rd7 Ke8 62. c8=Q+
END
Score: 1

[CMCanavessi]

Exactly the same game