search

herohde / morlock

Morlock chess engine
MIT License
14 stars 1 forks source link

unclear Turo ply count while mate #2

Closed tissatussa closed 1 year ago

tissatussa commented 1 year ago

hi,

using your Morlock Turo version, i just encountered a strange move by the engine, considering i set ply 6 in this game :

morlock-turo-ply6-dia1

here White (Turo ply 6) played Nf3xg5! which is a good move, making the white squares free for the Queen to give check and attack the opponent King .. Black took the kNight and we have this position :

morlock-turo-ply6-dia2

now i expected Turo to play Qh5!, justifying the kNight sacrifice, but it didn't : it played dxe5? .. and after Kf7 Qh5+ Kg7, Black survived the attack ..

but Qh5+ is mate-in-3, isn't it ? : Qh5+ Kd7 {the only move} Bh3+ g4 {again the only move} Bxg4# i count 6 plies, so why doesn't Morlock Turo find the mate ?

tissatussa commented 1 year ago

NOTE: i just discovered at ply 7 the Turo version DOES play Qh5+ and mates .. so it's my misunderstanding of plies ?

herohde commented 1 year ago

Thanks for reporting this issue. I would also expect it to find the mate at 6 ply (unless it ran out of time and cut it short or some such). Let me take a closer look -- do you have the PGN for the game? Or just the FEN?

tissatussa commented 1 year ago

the first diagram is after 10...e5

the second diagram is after 11. Nxg5 fxg5

[Event "engine vs engine"]
[Site "Holland"]
[Date "2023.08.12"]
[Round "?"]
[White "Morlock v0.89.1.0 Turo ply6"]
[Black "ARBE dp6 ORG"]
[Result "1/2-1/2"]
[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w - - 0 1"]
[GameDuration "00:11:04"]
[PlyCount "78"]
[SetUp "1"]
[TimeControl "1800"]

1. Nc3 {0.00/6 4.3s} Nc6 {2.7s} 2. d4 {0.00/6 5.3s} e6 {2.2s}
3. Nf3 {+0.01/6 25s} Bb4 {1.3s} 4. a3 {+0.43/6 8.7s} Bxc3+ {3.0s}
5. bxc3 {+1010.05/6 2.0s} d5 {1.6s} 6. Rg1 {+1010.10/6 18s} Qd6 {1.5s}
7. g3 {+1010.23/6 12s} f6 {6.3s} 8. a4 {+1010.31/6 15s} g5 {6.8s}
9. Ba3 {+1010.34/6 26s} Qd8 {4.3s} 10. e3 {+1010.35/6 7.3s} e5 {4.8s}
11. Nxg5 {+1040.19/6 20s} fxg5 {6.1s} 12. dxe5 {+M3/5 8.3s} Kf7 {13s}
13. Qh5+ {-1009.98/6 15s} Kg7 {4.0s} 14. f4 {-1010.09/6 20s} Bf5 {6.0s}
15. Rg2 {-1010.19/6 37s} a6 {10s} 16. fxg5 {-1009.88/6 45s} Qe8 {5.5s}
17. g4 {-1019.65/6 21s} Qxe5 {2.5s} 18. Kd2 {-1050.06/6 17s} Bg6 {12s}
19. Qh4 {-1030.20/6 23s} Re8 {12s} 20. Qg3 {-1020.96/6 26s} Qxg3 {16s}
21. Rxg3 {-1020.65/6 5.3s} Na5 {4.0s} 22. Bc5 {-1020.18/6 11s} Nc6 {3.4s}
23. h4 {-1020.33/6 18s} b6 {4.0s} 24. Ba3 {-1019.95/6 3.8s} Na5 {3.7s}
25. h5 {+0.01/6 6.0s} Be4 {1.6s} 26. Bxa6 {0.00/6 4.5s} Re5 {2.2s}
27. Rf1 {+1018.87/6 5.5s} c5 {3.7s} 28. h6+ {+0.09/6 4.6s} Kg6 {0.72s}
29. Rf8 {-1021.14/6 14s} Kxg5 {1.5s} 30. Bb2 {-1069.98/6 20s} Nc4+ {2.3s}
31. Kc1 {-1109.89/6 1.5s} Nxb2 {1.8s} 32. Kxb2 {-1109.97/6 3.0s} Kxh6 {1.5s}
33. Rb8 {-1109.95/6 5.2s} Kg5 {0.66s} 34. Rxb6 {-1059.97/6 3.4s} Nf6 {2.8s}
35. Be2 {-1060.75/6 4.6s} Ra8 {3.0s} 36. Rb5 {-1070.11/6 4.9s} Rc8 {3.7s}
37. Rb6 {-1060.77/6 9.4s} Ra8 {3.3s} 38. Rb5 {-1070.11/6 2.5s} Rc8 {3.8s}
39. Rb6 {-1060.77/6 6.2s} Ra8 {3.4s, Draw by 3-fold repetition} 1/2-1/2
tissatussa commented 1 year ago

btw. while trying to reproduce this game, i let the Black engine (my own script) play against your Morlock Turo depth 6 again, and it seems Turo does not always play the same moves .. eg. its first move may be 1.d4 or 1.Nc3 .. also in the above game, move 15 (Rg2) can be Kd2 .. why does Morlock Turo does not play the same moves all the time? It's a simple engine ..

tissatussa commented 1 year ago

btw. i did set a starting FEN, which is the normal starting position without castling rights, because "ARBE" v0.1 can not castle yet and it doesn't recognise such move made by the opponent ..

herohde commented 1 year ago

Re different moves: I added a bit of randomness to Turochamp (and Sargon) evaluation to prevent them being deterministic and thus more fun for repeated play imo. You can remove that by modifying the setup here: https://github.com/herohde/morlock/blob/main/cmd/turochamp/main.go#L45.

The problem seems to be with the mate-in-X computation with the quiescence search and transposition tables. If I disable the transposition tables, I get the correct move. The console mode makes it easier to try:

$ go run cmd/turochamp/main.go
> console 
> nohash
> r r1bqk1nr/ppp4p/2n5/3pp1p1/P2P4/B1P1P1P1/2P2P1P/R2QKBR1 w Qkq - 0 12

    a   b   c   d   e   f   g   h
  ---------------------------------
8 | r |   | b | q | k |   | n | r | 
  ---------------------------------
7 | p | p | p |   |   |   |   | p | 
  ---------------------------------
6 |   |   | n |   |   |   |   |   | 
  ---------------------------------
5 |   |   |   | p | p |   | p |   | 
  ---------------------------------
4 | P |   |   | P |   |   |   |   | 
  ---------------------------------
3 | B |   | P |   | P |   | P |   | 
  ---------------------------------
2 |   |   | P |   |   | P |   | P | 
  ---------------------------------
1 | R |   |   | Q | K | B | R |   | 
  ---------------------------------
    a   b   c   d   e   f   g   h

fen:    r1bqk1nr/ppp4p/2n5/3pp1p1/P2P4/B1P1P1P1/2P2P1P/R2QKBR1 w Qkq - 0 12
result: ?, ply: 1, hash: 0xeef152bde16731b2

> d 6
> a
depth=1 score=-1038.60 nodes=50 time=4.300906ms hash=0% pv=Qd1-h5
depth=2 score=-1039.40 nodes=345 time=21.411804ms hash=0% pv=Qd1-h5 Ke8-d7
depth=3 score=-1038.50 nodes=2402 time=65.261841ms hash=0% pv=Qd1-h5 Ke8-d7 Bf1-h3
depth=4 score=M5 nodes=49491 time=1.335480111s hash=0% pv=Qd1-h5 Ke8-d7 Bf1-h3 g5-g4
depth=5 score=M5 nodes=199584 time=5.045422388s hash=0% pv=Qd1-h5 Ke8-d7 Bf1-h3 g5-g4 Bh3*g4
depth=6 score=M5 nodes=3602132 time=1m27.349761742s hash=0% pv=Qd1-h5 Ke8-d7 Bf1-h3 g5-g4 Bh3*g4
bestmove Qd1-h5
Search, depth=6
  1. Qd1-h5 M5      (69474 nodes    pv Ke8-d7 Bf1-h3 g5-g4 Bh3*g4)
  2. f2-f4  -1010.21        (311843 nodes   pv h7-h5 d4*e5 Bc8-g4 Qd1-b1 Ra8-b8)
  3. d4*e5  -1039.58        (199740 nodes   pv Nc6*e5 Qd1-h5 Ne5-g6 Bf1-b5 c7-c6)
  4. h2-h4  -1039.79        (241903 nodes   pv h7-h5 Bf1-e2 g5-g4 d4*e5 Nc6*e5)
  5. Qd1-f3 -1039.82        (405633 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Bf1-b5 c7-c6)
  6. Bf1-b5 -1039.83        (54663 nodes    pv Qd8-f6 d4*e5 Qf6*e5 Qd1-h5 Ke8-d8)
  7. Bf1-g2 -1039.87        (223037 nodes   pv Ng8-e7 Qd1-h5 Ne7-g6 Ke1-d2 Qd8-f6)
  8. h2-h3  -1039.89        (141418 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Ba3-b4 Ke8-f7)
  9. Ba3-c5 -1039.91        (210300 nodes   pv Ng8-f6 d4*e5 Nc6*e5 a4-a5 Ke8-f7)
 10. Bf1-e2 -1039.92        (269673 nodes   pv Qd8-f6 Be2-h5 Ke8-d8 d4*e5 Qf6*e5)
 11. Qd1-e2 -1039.92        (169694 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Qe2-b5 Ke8-f7)
 12. Bf1-d3 -1039.92        (175309 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Bd3-b5 Ke8-f7)
 13. Rg1-g2 -1039.93        (143828 nodes   pv h7-h5 Bf1-e2 Ng8-f6 d4*e5 Nc6*e5)
 14. a4-a5  -1039.94        (193908 nodes   pv h7-h5 d4*e5 Nc6*e5 h2-h3 Ng8-e7)
 15. Rg1-h1 -1039.95        (220394 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Ba3-b4 Ke8-f7)
 16. Ra1-b1 -1040.01        (204191 nodes   pv Qd8-f6 d4*e5 Qf6*e5 Qd1-h5 Ke8-d8)
 17. Ba3-c1 -1040.05        (147643 nodes   pv Ng8-e7 Qd1-h5 Ne7-g6 Bc1-a3 Qd8-f6)
 18. Ba3-b2 -1040.05        (139512 nodes   pv h7-h5 d4*e5 Nc6*e5 Bb2-a3 Bc8-g4)
 19. Ra1-a2 -1040.11        (218333 nodes   pv Qd8-f6 d4*e5 Qf6*e5 Qd1-h5 Ke8-d8)
 20. e3-e4  -1040.11        (166723 nodes   pv h7-h5 Bf1-e2 Ng8-f6 e4*d5 Qd8*d5)
 21. Ra1-c1 -1040.12        (163964 nodes   pv h7-h5 d4*e5 Nc6*e5 Bf1-b5 c7-c6)
 22. Qd1-b1 -1040.16        (125701 nodes   pv e5*d4 c3*d4 Qd8-f6 Qb1-b5 Ng8-e7)
 23. Qd1-d3 -1040.16        (129544 nodes   pv e5*d4 c3*d4 Qd8-f6 Qd3-b5 Ng8-e7)
 24. Qd1-d2 -1040.16        (102947 nodes   pv Qd8-f6 Bf1-b5 Ng8-e7 d4*e5 Qf6*e5)
 25. g3-g4  -1040.18        (90576 nodes    pv e5*d4 c3*d4 Qd8-f6 Bf1-b5 Ng8-e7)
 26. f2-f3  -1040.21        (81617 nodes    pv e5*d4 c3*d4 Qd8-f6 Bf1-b5 Ng8-e7)
 27. Qd1-c1 -1040.27        (101933 nodes   pv Qd8-f6 Bf1-b5 e5*d4 c3*d4 Ng8-e7)
 28. Ke1-d2 -1040.35        (202397 nodes   pv Qd8-f6 Qd1-e1 Ng8-e7 Bf1-b5 Bc8-d7)
 29. Ke1-e2 -1040.58        (106035 nodes   pv e5*d4 c3*d4 Qd8-f6 Ke2-e1 Ng8-e7)
 30. c3-c4  -1040.81        (194569 nodes   pv g5-g4 c4*d5 Qd8*d5 c2-c4 Qd5-a5)
 31. Bf1-h3 -1049.76        (98291 nodes    pv Bc8*h3 Qd1-h5 Ke8-d7 Qh5*h3 Kd7-e8)
 32. Ba3-b4 -1069.51        (84190 nodes    pv Nc6*b4 c3*b4 e5*d4 Bf1-b5 c7-c6)
 33. Bf1-c4 -1129.90        (137836 nodes   pv d5*c4 Qd1-h5 Ke8-d7 Qh5-f7 Ng8-e7)
 34. Bf1-a6 -1159.72        (75964 nodes    pv b7*a6 Qd1-h5 Ke8-d7 d4*e5 Nc6*e5)
 35. Ba3-f8 -1160.05        (82640 nodes    pv Ke8*f8 Qd1-f3 Kf8-g7 d4*e5 Nc6*e5)
 36. Ba3-d6 -1160.21        (89358 nodes    pv Qd8*d6 d4*e5 Qd6*e5 Qd1-h5 Ke8-d8)
 37. Ba3-e7 -1160.27        (70939 nodes    pv Ng8*e7 Qd1-h5 Ne7-g6 d4*e5 Nc6*e5)
 38. Qd1-g4 -1480.21        (20485 nodes    pv Bc8*g4 h2-h3 Bg4-f5 d4*e5 Bf5*c2)

Turochamp searches captures as a "considerable move", so it finds the mate at depth 4. A deeper search can't improve on a mate in 5 ply (M5).

I can repro the poor move, if I do a prior search from r1bqk1nr/ppp4p/2n2p2/3pp1p1/P2P4/B1P1PNP1/2P2P1P/R2QKBR1 w Qkq - 0 11 first with transposition tables. Then I get an incorrect mate in 3 ply (M3) with the bad move you saw in your game (it's in the PGN as well: 12. dxe5 {+M3/5 8.3s} Kf7 {13s}), yet it is not confirmed by the breakdown:

    a   b   c   d   e   f   g   h
  ---------------------------------
8 | r |   | b | q | k |   | n | r | 
  ---------------------------------
7 | p | p | p |   |   |   |   | p | 
  ---------------------------------
6 |   |   | n |   |   |   |   |   | 
  ---------------------------------
5 |   |   |   | p | p |   | p |   | 
  ---------------------------------
4 | P |   |   | P |   |   |   |   | 
  ---------------------------------
3 | B |   | P |   | P |   | P |   | 
  ---------------------------------
2 |   |   | P |   |   | P |   | P | 
  ---------------------------------
1 | R |   |   | Q | K | B | R |   | 
  ---------------------------------
    a   b   c   d   e   f   g   h

fen:    r1bqk1nr/ppp4p/2n5/3pp1p1/P2P4/B1P1P1P1/2P2P1P/R2QKBR1 w Qkq - 0 12
result: ?, ply: 3, hash: 0xeef152bde16731b2

a
depth=1 score=-1038.60 nodes=43 time=2.609044ms hash=1% pv=Qd1-h5
depth=2 score=-1039.17 nodes=223 time=12.552893ms hash=1% pv=Qd1-e2 e5*d4
depth=3 score=-1038.50 nodes=1922 time=52.052052ms hash=1% pv=Qd1-h5 Ke8-d7 Bf1-h3
depth=4 score=M5 nodes=6412 time=148.789079ms hash=1% pv=Qd1-h5 Ke8-d7 Bf1-h3 g5-g4
depth=5 score=M3 nodes=254302 time=5.721763514s hash=2% pv=d4*e5 Bc8-d7 Qd1-h5.             <-------------
bestmove d4*e5
Search, depth=5
  1. Qd1-h5 M5      (4743 nodes pv Ke8-d7 Bf1-h3 g5-g4 Bh3*g4)
  2. d4*e5  -1038.61        (28778 nodes    pv Nc6*e5 Bf1-b5 c7-c6 Qd1-h5)
  3. Bf1-e2 -1038.90        (29016 nodes    pv Ng8-e7 d4*e5 Nc6*e5 Be2-h5)
  4. Bf1-b5 -1038.95        (11415 nodes    pv e5*d4 e3*d4 Ng8-e7 Qd1-h5)
  5. Qd1-e2 -1038.97        (17224 nodes    pv e5*d4 e3*d4 Ng8-e7 Qe2-h5)
  6. h2-h3  -1039.20        (15396 nodes    pv e5*d4 e3*d4 h7-h5 Qd1-e2)
  7. f2-f4  -1039.33        (19517 nodes    pv e5*f4 Qd1-h5 Ke8-d7 g3*f4)
  8. h2-h4  -1039.62        (15883 nodes    pv h7-h5 d4*e5 g5*h4 Bf1-b5)
  9. Ba3-c5 -1039.67        (22798 nodes    pv Ng8-f6 d4*e5 Nf6-g4 f2-f4)
 10. Qd1-f3 -1039.67        (25524 nodes    pv Ng8-f6 d4*e5 Nf6-g4 Bf1-b5)
 11. Bf1-g2 -1039.68        (15898 nodes    pv Ng8-f6 d4*e5 Nc6*e5 Qd1-d4)
 12. Bf1-d3 -1039.72        (32883 nodes    pv Ng8-f6 d4*e5 Nf6-g4 Bd3-b5)
 13. Rg1-g2 -1039.77        (21264 nodes    pv Ng8-f6 d4*e5 Nf6-g4 f2-f4)
 14. Rg1-h1 -1039.78        (20417 nodes    pv Ng8-f6 d4*e5 Nf6-g4 Bf1-b5)
 15. Qd1-d3 -1039.79        (21079 nodes    pv e5*d4 e3*d4 Ng8-e7 Qd3-e3)
 16. Qd1-d2 -1039.81        (17472 nodes    pv Ng8-f6 Bf1-b5 Nf6-e4 Qd2-d3)
 17. Qd1-b1 -1039.81        (13905 nodes    pv e5*d4 c3*d4 Ng8-e7 Qb1-b5)
 18. Ra1-b1 -1039.83        (20703 nodes    pv Ng8-f6 d4*e5 Nf6-g4 Bf1-b5)
 19. Qd1-c1 -1039.89        (20422 nodes    pv Qd8-f6 d4*e5 Qf6*e5 Qc1-b2)
 20. c3-c4  -1039.92        (13705 nodes    pv h7-h5 Qd1-f3 Ng8-f6 d4*e5)
 21. Ra1-a2 -1039.93        (20028 nodes    pv Ng8-f6 d4*e5 Nf6-g4 Bf1-b5)
 22. f2-f3  -1039.94        (10110 nodes    pv Qd8-f6 d4*e5 Qf6*e5 Qd1-d2)
 23. Ba3-b2 -1039.96        (18082 nodes    pv Ng8-f6 d4*e5 Nf6-g4 Bb2-a3)
 24. Ba3-c1 -1039.96        (14563 nodes    pv Ng8-f6 d4*e5 Nf6-g4 Bc1-a3)
 25. Ra1-c1 -1039.96        (17201 nodes    pv Ng8-f6 d4*e5 Nf6-g4 Bf1-b5)
 26. g3-g4  -1040.01        (14960 nodes    pv Qd8-f6 Bf1-b5 Ng8-e7 Qd1-d3)
 27. Ke1-d2 -1040.01        (26420 nodes    pv Ng8-f6 d4*e5 Nf6-e4 Kd2-e1)
 28. Ke1-e2 -1040.18        (8030 nodes pv e5*d4 c3*d4 Ng8-e7 Qd1-d3)
 29. Bf1-h3 -1048.94        (14635 nodes    pv Bc8*h3 Qd1-h5 Ke8-d7 Qh5*h3)
 30. Ba3-b4 -1068.50        (12306 nodes    pv e5*d4 Qd1-h5 Ke8-d7 Bf1-h3)
 31. a4-a5  -1068.53        (8147 nodes pv e5*d4 Qd1-h5 Ke8-d7 Bf1-h3)
 32. e3-e4  -1068.54        (10187 nodes    pv d5*e4 Qd1-h5 Ke8-d7 Bf1-h3)
 33. Bf1-a6 -1159.00        (7674 nodes pv b7*a6 Qd1-h5 Ke8-d7 Qh5-g4)
 34. Bf1-c4 -1159.01        (8972 nodes pv d5*c4 Qd1-h5 Ke8-d7 Qh5-g4)
 35. Ba3-d6 -1159.16        (8048 nodes pv Qd8*d6 d4*e5 Nc6*e5 Qd1-h5)
 36. Ba3-f8 -1159.80        (7892 nodes pv Ke8*f8 d4*e5 Nc6*e5 Qd1-d4)
 37. Ba3-e7 -1159.82        (10698 nodes    pv Ke8*e7 d4*e5 Nc6*e5 Qd1-d4)
 38. Qd1-g4 -1460.30        (4722 nodes pv Bc8*g4 h2-h3 Bg4-f5 Ke1-d2)

I'll look closer into that bug. Thanks for reporting it.

tissatussa commented 1 year ago

Thanks for your time to investigate this, i thought you could be curious to find the cause of this Issue, because you're the creator of this script.

Indeed randomness is more fun for repeated play, but I have no objection when your TuroChamp version strictly implements the original evaluation rule and thus plays the same moves all the time. Users might even expect that, eg. for evaluation of those original rules in certain positions. Therefor i suggest you create an UCI option for a parameter like "add some randomness" ? I know such UCI (checkbox, selectbox, etc) options are easy to code, and CuteChess nicely displays them in its settings pane. Other UCI options might be Depth, logging On/Off, and even selecting a different set of evaluation values ?

herohde commented 1 year ago

Appreciate it. Thanks for the suggestion. Depth and Noise (= evaluation randomness) are now UCI options for convenience.

tissatussa commented 1 year ago

..Depth and Noise (= evaluation randomness) are now UCI options..

i really like your newest v0.89.2.0 : those UCI options for Depth and Noise are awesome ! at first i built the Turochamp engine .. to make it work, i had to use the gvm tool (i'm on Linux) to install and set a proper GO version : i got a valid binary of 2.7 Mb :

uci
id name TUROCHAMP (1948) 0.89.2.0.nohash
id author Alan Turing and David Champernowne
option name Depth type spin default 2 min 0 max 100
option name Hash type spin default 0 min 0 max 16384
option name Noise type spin default 10 min 0 max 10000
uciok

then i did some tests in CuteChess, using 13 min + 10 sec bonus, setting Depth 6, 7 and even 8, and Turochamp played well against some (weak) engines .. i see Noise can be set 0 upto 10000, but why default 10 ? what does it mean ? i set Noise 5000 (the half of your max) and just watch the games evolving : no obvious mistakes were made, so i guess that Noise feature picks moves which "are also" good - or not bad : how does Noise work, can you elaborate on this ? does this feature still resemble the Turochamp principe ?

about the Depth option : using your previous v0.89.1.0 i had to build binaries with custom depth by adjusting this value in the source code .. now this option can be set in the (CuteChess) GUI options pane, which indeed is very convenient .. with my time control, Turochamp seems to keep its move-time below 30 seconds, even with Depth 8 : so i guess some max-time-per-move is used ? and why can Turochamp only reach higher depths after a lot of time, unlike many other engines ? does it have simple / poor pruning ? -- respect though..

these are just some questions and remarks. can you tell us your ideas to implement these Morlock engines, esp. the Turochamp version ? Btw. i like the original Turochamp PDF document!

herohde commented 1 year ago

Glad that it was helpful!

Re noise: The engine uses an evaluation score in "pawns" as a nominal value. The noise adds a randomness in milli-pawns. So 1,000 noise means +/- 0.5 pawns. The UCI option unfortunately does not allows an inline description to these options.

However, as you can tell from reading the original paper, Turochamp does not use "pawns" as a scoring value, but rather the tuple of (material ratio, position play). The implementation here combines them into a "pawns" score but it is in reality not in pawns: see https://github.com/herohde/morlock/blob/main/cmd/turochamp/turochamp/eval.go#L20. The scoring is also made more symmetrical to make it more stable and work with odd search depths. The (arbitrary) UCI maximum of 10k is indeed small compared to the material ratio. But larger noise will drown out position play.

Re depth: The time control assumes that there are 40 moves left in the game to manage time, see https://github.com/herohde/morlock/blob/main/pkg/search/searchctl/timectrl.go#L26. This simple heuristic makes it hard to run out of time, but is overly conservative later in the game.

The search implementation is indeed not very sophisticated. I found it more enjoyable to to focus on the low depths, where search does not hide the engines' quirks. At low ply, search efficiency does not matter. Also, Sargon has some score factors that are relative to the root position, which makes transposition table scores unreliable to reuse from past moves. In other words, these old engines have some features that do not fit easily into modern search optimizations. Turochamp specifically uses an unbounded quiescence search of "considerable moves", which may be expensive. Sargon similarly has a non-standard quiescence search heuristic.

Not sure I understand your last question. Can you elaborate?

tissatussa commented 1 year ago

thanks for the extensive reply and insights, it's also valuable info for other users, reading about this (i could have created a seperate Issue for it?).

my last question .. you mean tell us your ideas to implement these Morlock engines, esp. the Turochamp version ? well, i guess Turin didn't use recursion upto depth 6 or so, while calculating on paper .. so did you add some own ideas into this, eg. handling odd plies ? (i see the node counts in CuteChess sometimes show very low -30 or so- at some odd depth ? while normally the node count always goes up while reaching higher depths.

i also have my own ideas .. i could tweak the evaluation (not the search) with some own factors, like PSTs .. i'm not familiar with GO though .. maybe i should use the Nim version, a language i learned lately .. btw. i must yet read those linked docs.

herohde commented 1 year ago

Gotcha. My goal with Turochamp (and Sargon) was to implement it as faithfully as possible to the paper, but in a more standard engine framework. The only tweak I really made was to make the position-play evaluation symmetrical and fit the heuristics into a signed "pawns" score. The symmetry tweak was to counter some overly dubious moves at ply 1 that made it less fun, if I recall correctly, not improve the evaluation per se.

tissatussa commented 1 year ago

overall your explanation is clear to me, and valuable ! Sounds like you did a lot of experiments and tests to get proper results. Btw. for now i only focus on the Turochamp engine, i understand those Rules Turin described in his paper and i think i agree .. i'm rather surprised how well your version plays with higher depth setting - myself i can hardly win .. after my tests i tend to conclude its rating is about 1300 - me 1800 !?

about the "pawns" score : it seems (when using Noise 5000) the engine gives eval values upto around +-0.6 (cp?) and often at some point those figures get 1000 upto 5000 (when winning) .. this is curious : did my Noise 5000 cause these values ? Is Noise some treshold ? i don't fully understand these values, i expect 0.6 will be followed by 1.0 upto 5.0 in that case, or is this a mis-interpretation of the CuteChess GUI ?

by threshold i mean : when Noise is high, even a move with an eval less then that threshold could become bestmove ? Is your Noise value some "Contempt", as used by other authors, or a different concept ? In that case we could try to construct something like "Advisors", giving their own stats which can influence bestmove when the eval of their suggestion is within that treshold ?

this is my idea in a nutshell .. are you willing to discuss such further ? i can open an Issue for it.

herohde commented 1 year ago

I realized that I broke the noise functionality when I moved it around for UCI. The noise option had no effect. That's now fixed. Sorry about causing confusion.

The score values for Turochamp don't fit the usual pawn interpretation. For example, black to move with the white queen en prise:

    a   b   c   d   e   f   g   h
  ---------------------------------
8 | r | n | b | q | k | b | n | r | 
  ---------------------------------
7 | p | p | p |   | p | p | p | p | 
  ---------------------------------
6 |   |   |   | p |   |   |   |   | 
  ---------------------------------
5 |   |   |   |   |   |   |   |   | 
  ---------------------------------
4 |   |   |   |   |   |   | Q |   | 
  ---------------------------------
3 |   |   |   |   | P |   |   |   | 
  ---------------------------------
2 | P | P | P | P |   | P | P | P | 
  ---------------------------------
1 | R | N | B |   | K | B | N | R | 
  ---------------------------------
    a   b   c   d   e   f   g   h

fen:    rnbqkbnr/ppp1pppp/3p4/8/6Q1/4P3/PPPP1PPP/RNB1KBNR b KQkq - 1 2
result: ?, ply: 4, hash: 0x65bd7a29073de7e7

a 1
depth=1 score=1320.34 nodes=27 time=3.039988ms hash=0% pv=Bc8*g4
bestmove Bc8*g4
Search, depth=1
  1. Bc8*g4 1320.34     (1 nodes    pv )
  2. e7-e5  -0.09       (1 nodes    pv )
  3. Nb8-c6 -0.11       (1 nodes    pv )
  4. e7-e6  -0.12       (1 nodes    pv )
  5. Bc8-e6 -0.13       (1 nodes    pv )
  6. Bc8-d7 -0.13       (1 nodes    pv )
  7. Ng8-f6 -0.14       (1 nodes    pv )
  8. Ng8-h6 -0.20       (1 nodes    pv )
  9. c7-c6  -0.22       (1 nodes    pv )
 10. a7-a5  -0.23       (1 nodes    pv )
 11. h7-h5  -0.24       (1 nodes    pv )
 12. Nb8-d7 -0.25       (1 nodes    pv )
 13. Qd8-d7 -0.27       (1 nodes    pv )
 14. g7-g6  -0.27       (1 nodes    pv )
 15. Nb8-a6 -0.28       (1 nodes    pv )
 16. c7-c5  -0.30       (1 nodes    pv )
 17. h7-h6  -0.33       (1 nodes    pv )
 18. a7-a6  -0.34       (1 nodes    pv )
 19. f7-f5  -0.34       (1 nodes    pv )
 20. d6-d5  -0.35       (1 nodes    pv )
 21. b7-b6  -0.38       (1 nodes    pv )
 22. f7-f6  -0.39       (1 nodes    pv )
 23. g7-g5  -1020.20        (2 nodes    pv )
 24. b7-b5  -1020.75        (2 nodes    pv )
 25. Bc8-f5 -1090.70        (2 nodes    pv )

The scores are from blacks point of view in pawns, where positive is ahead. Taking the queen is vastly better than any other move, but obviously not worth more than 1000 pawns (most engines would score it as ~9 pawns ahead). That is just an artifact of combining "material ratio" and "position play" difference into a single score, where the ratio always dominates. The ratio of 1.32 is encoded as 1320 pawns and the position play score of 3.4 is encoded as 0.34. Even losing a single pawn with g7-g5, for example, bring a losing -1.02 ration -- expressed as -1020 "pawns". This is where the modern emphasis on scoring everything in "pawns" clash with Turing's design.

Back to noise: the values are quite close -- they are often just a few centi-pawn apart the way the score is combined. The noise random element just aims vary the play. It's not contempt or anything clever. If noise is high enough, any move can be the best move and the play is random. The starting position:

    a   b   c   d   e   f   g   h
  ---------------------------------
8 | r | n | b | q | k | b | n | r | 
  ---------------------------------
7 | p | p | p | p | p | p | p | p | 
  ---------------------------------
6 |   |   |   |   |   |   |   |   | 
  ---------------------------------
5 |   |   |   |   |   |   |   |   | 
  ---------------------------------
4 |   |   |   |   |   |   |   |   | 
  ---------------------------------
3 |   |   |   |   |   |   |   |   | 
  ---------------------------------
2 | P | P | P | P | P | P | P | P | 
  ---------------------------------
1 | R | N | B | Q | K | B | N | R | 
  ---------------------------------
    a   b   c   d   e   f   g   h

fen:    rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
result: ?, ply: 1, hash: 0xc73b6db27699f0b0

a 1
depth=1 score=0.44 nodes=21 time=994.935µs hash=0% pv=e2-e3
bestmove e2-e3
Search, depth=1
  1. e2-e3  0.44        (1 nodes    pv )
  2. e2-e4  0.42        (1 nodes    pv )
  3. d2-d4  0.33        (1 nodes    pv )
  4. d2-d3  0.27        (1 nodes    pv )
  5. Ng1-f3 0.23        (1 nodes    pv )
  6. Nb1-c3 0.23        (1 nodes    pv )
  7. h2-h4  0.18        (1 nodes    pv )
  8. a2-a4  0.18        (1 nodes    pv )
  9. Ng1-h3 0.18        (1 nodes    pv )
 10. c2-c4  0.18        (1 nodes    pv )
 11. Nb1-a3 0.18        (1 nodes    pv )
 12. c2-c3  0.15        (1 nodes    pv )
 13. b2-b4  0.15        (1 nodes    pv )
 14. g2-g4  0.15        (1 nodes    pv )
 15. g2-g3  0.13        (1 nodes    pv )
 16. b2-b3  0.13        (1 nodes    pv )
 17. h2-h3  0.08        (1 nodes    pv )
 18. a2-a3  0.08        (1 nodes    pv )
 19. f2-f4  -0.06       (1 nodes    pv )
 20. f2-f3  -0.09       (1 nodes    pv )

Using noise of 100 (= +/- 5 centipawns), for example, either e2-e3 or e2-e4 could be the initial best move. (Note that the console mode breakdown does not use noise or transposition tables, even if the actual search does.)

tissatussa commented 1 year ago

regarding Turochamp : in main.go line 24 i see :

usage: turochamp [options]

how can we see this output in console ? you showed some output diagrams and info, regarding "console mode breakdown" .. what is that ? How can i reproduce your output ? I just did uci, and commands like go depth #, but eg. help, info, display etc. seem invalid commands, indeed non-UCI afaik. Which [options] do we have ? Btw. i expected the code of the function flag.PrintDefaults() would give me answers, but it's not there, being imported by "flag" !? (i'm not into GO programming)

herohde commented 1 year ago

You can see the command line options by passing -h as a command line argument:

$ go run cmd/turochamp/main.go -h
usage: turochamp [options]

TUROCHAMP is a re-implementation of Alan Turing and David Champernowne's 1948
chess engine, described in "Digital computers applied to games" (1953). The
re-implementation uses the UCI protocol for use in modern chess programs.
Options:
  -alsologtostderr
        log to standard error as well as files
  -log_backtrace_at value
        when logging hits line file:N, emit a stack trace
  -log_dir string
        If non-empty, write log files in this directory
  -logtostderr
        log to standard error instead of files
  -noise uint
        Evaluation noise in "millipawns" (zero if deterministic) (default 10)
  -ply uint
        Search depth limit (zero if no limit) (default 2)
  -stderrthreshold value
        logs at or above this threshold go to stderr
  -v value
        log level for V logs
  -vmodule value
        comma-separated list of pattern=N settings for file-filtered logging

You can set the noise and ply defaults via the command line. The other are logging settings.

You can use the console mode by passing console instead of uci as the first input. It's mainly for internal development use and supports various commands as defined here: https://github.com/herohde/morlock/blob/main/pkg/engine/console/console.go#L74.

In a previous comment, I indicated some commands in that mode with >:

$ go run cmd/turochamp/main.go
> console 
> r r1bqk1nr/ppp4p/2n5/3pp1p1/P2P4/B1P1P1P1/2P2P1P/R2QKBR1 w Qkq - 0 12

    a   b   c   d   e   f   g   h
  ---------------------------------
8 | r |   | b | q | k |   | n | r | 
  ---------------------------------
7 | p | p | p |   |   |   |   | p | 
  ---------------------------------
6 |   |   | n |   |   |   |   |   | 
  ---------------------------------
5 |   |   |   | p | p |   | p |   | 
  ---------------------------------
4 | P |   |   | P |   |   |   |   | 
  ---------------------------------
3 | B |   | P |   | P |   | P |   | 
  ---------------------------------
2 |   |   | P |   |   | P |   | P | 
  ---------------------------------
1 | R |   |   | Q | K | B | R |   | 
  ---------------------------------
    a   b   c   d   e   f   g   h

fen:    r1bqk1nr/ppp4p/2n5/3pp1p1/P2P4/B1P1P1P1/2P2P1P/R2QKBR1 w Qkq - 0 12
result: ?, ply: 1, hash: 0xeef152bde16731b2

> d 6
> a
depth=1 score=-1038.60 nodes=50 time=4.300906ms hash=0% pv=Qd1-h5
depth=2 score=-1039.40 nodes=345 time=21.411804ms hash=0% pv=Qd1-h5 Ke8-d7
depth=3 score=-1038.50 nodes=2402 time=65.261841ms hash=0% pv=Qd1-h5 Ke8-d7 Bf1-h3
depth=4 score=M5 nodes=49491 time=1.335480111s hash=0% pv=Qd1-h5 Ke8-d7 Bf1-h3 g5-g4
depth=5 score=M5 nodes=199584 time=5.045422388s hash=0% pv=Qd1-h5 Ke8-d7 Bf1-h3 g5-g4 Bh3*g4
depth=6 score=M5 nodes=3602132 time=1m27.349761742s hash=0% pv=Qd1-h5 Ke8-d7 Bf1-h3 g5-g4 Bh3*g4
bestmove Qd1-h5
Search, depth=6
  1. Qd1-h5 M5      (69474 nodes    pv Ke8-d7 Bf1-h3 g5-g4 Bh3*g4)
  2. f2-f4  -1010.21        (311843 nodes   pv h7-h5 d4*e5 Bc8-g4 Qd1-b1 Ra8-b8)
  3. d4*e5  -1039.58        (199740 nodes   pv Nc6*e5 Qd1-h5 Ne5-g6 Bf1-b5 c7-c6)
  4. h2-h4  -1039.79        (241903 nodes   pv h7-h5 Bf1-e2 g5-g4 d4*e5 Nc6*e5)
  5. Qd1-f3 -1039.82        (405633 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Bf1-b5 c7-c6)
  6. Bf1-b5 -1039.83        (54663 nodes    pv Qd8-f6 d4*e5 Qf6*e5 Qd1-h5 Ke8-d8)
  7. Bf1-g2 -1039.87        (223037 nodes   pv Ng8-e7 Qd1-h5 Ne7-g6 Ke1-d2 Qd8-f6)
  8. h2-h3  -1039.89        (141418 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Ba3-b4 Ke8-f7)
  9. Ba3-c5 -1039.91        (210300 nodes   pv Ng8-f6 d4*e5 Nc6*e5 a4-a5 Ke8-f7)
 10. Bf1-e2 -1039.92        (269673 nodes   pv Qd8-f6 Be2-h5 Ke8-d8 d4*e5 Qf6*e5)
 11. Qd1-e2 -1039.92        (169694 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Qe2-b5 Ke8-f7)
 12. Bf1-d3 -1039.92        (175309 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Bd3-b5 Ke8-f7)
 13. Rg1-g2 -1039.93        (143828 nodes   pv h7-h5 Bf1-e2 Ng8-f6 d4*e5 Nc6*e5)
 14. a4-a5  -1039.94        (193908 nodes   pv h7-h5 d4*e5 Nc6*e5 h2-h3 Ng8-e7)
 15. Rg1-h1 -1039.95        (220394 nodes   pv Ng8-f6 d4*e5 Nc6*e5 Ba3-b4 Ke8-f7)
 16. Ra1-b1 -1040.01        (204191 nodes   pv Qd8-f6 d4*e5 Qf6*e5 Qd1-h5 Ke8-d8)
 17. Ba3-c1 -1040.05        (147643 nodes   pv Ng8-e7 Qd1-h5 Ne7-g6 Bc1-a3 Qd8-f6)
 18. Ba3-b2 -1040.05        (139512 nodes   pv h7-h5 d4*e5 Nc6*e5 Bb2-a3 Bc8-g4)
 19. Ra1-a2 -1040.11        (218333 nodes   pv Qd8-f6 d4*e5 Qf6*e5 Qd1-h5 Ke8-d8)
 20. e3-e4  -1040.11        (166723 nodes   pv h7-h5 Bf1-e2 Ng8-f6 e4*d5 Qd8*d5)
 21. Ra1-c1 -1040.12        (163964 nodes   pv h7-h5 d4*e5 Nc6*e5 Bf1-b5 c7-c6)
 22. Qd1-b1 -1040.16        (125701 nodes   pv e5*d4 c3*d4 Qd8-f6 Qb1-b5 Ng8-e7)
 23. Qd1-d3 -1040.16        (129544 nodes   pv e5*d4 c3*d4 Qd8-f6 Qd3-b5 Ng8-e7)
 24. Qd1-d2 -1040.16        (102947 nodes   pv Qd8-f6 Bf1-b5 Ng8-e7 d4*e5 Qf6*e5)
 25. g3-g4  -1040.18        (90576 nodes    pv e5*d4 c3*d4 Qd8-f6 Bf1-b5 Ng8-e7)
 26. f2-f3  -1040.21        (81617 nodes    pv e5*d4 c3*d4 Qd8-f6 Bf1-b5 Ng8-e7)
 27. Qd1-c1 -1040.27        (101933 nodes   pv Qd8-f6 Bf1-b5 e5*d4 c3*d4 Ng8-e7)
 28. Ke1-d2 -1040.35        (202397 nodes   pv Qd8-f6 Qd1-e1 Ng8-e7 Bf1-b5 Bc8-d7)
 29. Ke1-e2 -1040.58        (106035 nodes   pv e5*d4 c3*d4 Qd8-f6 Ke2-e1 Ng8-e7)
 30. c3-c4  -1040.81        (194569 nodes   pv g5-g4 c4*d5 Qd8*d5 c2-c4 Qd5-a5)
 31. Bf1-h3 -1049.76        (98291 nodes    pv Bc8*h3 Qd1-h5 Ke8-d7 Qh5*h3 Kd7-e8)
 32. Ba3-b4 -1069.51        (84190 nodes    pv Nc6*b4 c3*b4 e5*d4 Bf1-b5 c7-c6)
 33. Bf1-c4 -1129.90        (137836 nodes   pv d5*c4 Qd1-h5 Ke8-d7 Qh5-f7 Ng8-e7)
 34. Bf1-a6 -1159.72        (75964 nodes    pv b7*a6 Qd1-h5 Ke8-d7 d4*e5 Nc6*e5)
 35. Ba3-f8 -1160.05        (82640 nodes    pv Ke8*f8 Qd1-f3 Kf8-g7 d4*e5 Nc6*e5)
 36. Ba3-d6 -1160.21        (89358 nodes    pv Qd8*d6 d4*e5 Qd6*e5 Qd1-h5 Ke8-d8)
 37. Ba3-e7 -1160.27        (70939 nodes    pv Ng8*e7 Qd1-h5 Ne7-g6 d4*e5 Nc6*e5)
 38. Qd1-g4 -1480.21        (20485 nodes    pv Bc8*g4 h2-h3 Bg4-f5 d4*e5 Bf5*c2)
tissatussa commented 1 year ago

nice ! this is exactly what i hoped for .. so "a" is "analyse" .. that explains your output, thanks!

[ i decided to close this Issue now, our info tends to go beyond the subject ]