v0.1.2 can find bestmove but didn't

[tissatussa]

in this position Eccat v0.1.2 made the wrong move a4? and it didn't recover, it lost the game :

1r1r2k1/1bbp1p2/2p2p1p/p1q5/4PP2/PPN2B1P/1K1Q2P1/3RR3 w - - 1 23

however, when i let Eccat v0.1.2 think for a long time, it gives another bestmove :

while analysing, the move a4? is found after 932 ms (upto depth 9), which is much faster than it did in the game : a4 was played after 11 seconds !? (depth 9) how is this possible ? Some cache / hash problem ?

for reference, here's the Stockfish analysis of this position :

[ i'm on Xubuntu 22.04 ]

[skycloudd]

On my current local version i get the following

UCI output

``` eccat v0.1.2 (6e89ee6-dirty) by skycloudd (main, 2024-05-14) [Rust 1.78.0] on Darwin position fen 1r1r2k1/1bbp1p2/2p2p1p/p1q5/4PP2/PPN2B1P/1K1Q2P1/3RR3 w - - 1 23 go infinite info depth 1 seldepth 6 time 0 score cp 52 nodes 105 nps 178660 hashfull 0 pv f4f5 info depth 2 seldepth 6 time 1 score cp 34 nodes 762 nps 623801 hashfull 0 pv a3a4 b8c8 info depth 3 seldepth 9 time 2 score cp 51 nodes 4742 nps 1661430 hashfull 0 pv a3a4 c5b6 f4f5 info depth 4 seldepth 11 time 4 score cp 37 nodes 8711 nps 1823402 hashfull 0 pv a3a4 f6f5 b2b1 c7f4 d2f4 c5c3 f4h6 f5e4 f3e4 c3b3 info depth 5 seldepth 12 time 10 score cp 48 nodes 24357 nps 2402021 hashfull 0 pv a3a4 c5b6 e4e5 f6e5 f4e5 info depth 6 seldepth 17 time 21 score cp 35 nodes 49243 nps 2272019 hashfull 2 pv a3a4 c7b6 b2b1 c5f2 d2d6 g8g7 info depth 7 seldepth 17 time 59 score cp 39 nodes 142509 nps 2395508 hashfull 5 pv h3h4 c5e7 f3g4 d7d6 f4f5 g8g7 b2b1 info depth 8 seldepth 18 time 212 score cp 39 nodes 515400 nps 2427240 hashfull 20 pv h3h4 c5e7 info depth 9 seldepth 22 time 1345 score cp 35 nodes 3390782 nps 2519316 hashfull 138 pv d1c1 c7b6 f4f5 c5f2 e1e2 f2h4 c1d1 b8c8 b2b1 info depth 10 seldepth 27 time 8206 score cp 25 nodes 19202057 nps 2339803 hashfull 665 pv c3a4 c5e7 g2g3 d7d6 a4c3 d8e8 d2c2 b8d8 b2b1 c7b6 info depth 11 seldepth 27 time 17815 score cp 24 nodes 42970654 nps 2411984 hashfull 956 pv c3a4 c5f8 a4c3 f8g7 g2g4 b8c8 b2b1 d7d6 f4f5 g7g5 d2f2 info depth 12 seldepth 27 time 35180 score cp 24 nodes 83006000 nps 2359452 hashfull 999 pv c3a4 c5e7 b2a2 b7a6 d2e3 a6b7 f3h5 g8h8 a4b6 d7d5 h5f3 c7b6 e3b6 d5e4 e1e4 ```

It starts off with the same a4 move but at depth 10 finally decides that Na4 is the best move (which the Lichess Stockfish analysis seems to at least agree with):

I'm admittedly having trouble myself understanding this position (I'm not that good at chess :p) , but from what I can tell here it seems like eccat just isn't fast enough (yet) to see deep enough.

while analysing, the move a4? is found after 932 ms (upto depth 9), which is much faster than it did in the game : a4 was played after 11 seconds !? (depth 9) how is this possible ? Some cache / hash problem ?

I'm not sure what it could be. Were you using the same hash size? (eccats default is 16mb, but your gui may have changed it?) Or were you running multiple games simultaneously?

Thanks for reporting this anyway :) I hope this slowly gets fixed as i optimise it more and improve the evaluation function. (and I should really start doing some more rigorous code testing as well)

[tissatussa]

I'm admittedly having trouble myself understanding this position (I'm not that good at chess :p)

aha, good to know .. maybe you can interpret my explanation of this position into some evaluation values / HCE ..

but from what I can tell here it seems like eccat just isn't fast enough (yet) to see deep enough.

that's true, but if your HCE is smart, it knows about pawn structures .. here the move a4 weakens the back squares, it can't be undone, pawn can't go backwards .. so, pawns should only be pushed "when needed" .. i guess some bonus can be given (or yet invented) to suit this issue, but no rule in chess is always true, i feel..

...how is this possible ? Some cache / hash problem ? I'm not sure what it could be. Were you using the same hash size? (eccats default is 16mb, but your gui may have changed it?) Or were you running multiple games simultaneously?

i always run only 1 game in CuteChess, after that i can do another .. i set 128 Mb Hash for all engines.

Thanks for reporting this anyway :) I hope this slowly gets fixed as i optimise it more and improve the evaluation function. (and I should really start doing some more rigorous code testing as well)

i did many "studies" by feeding special FENs to (weak) engines to see if they find (play) the 'solution' .. these are positions with clearly 1 best move, although hard to find for most of them, sometimes even 3000+ can't find such puzzle .. writing your own HCE only by tuning and testing (self play) without knowing the existing parameters (=chess basics) won't reach far ..

many things are involved developing a chess engine .. keep it simple, first create a fundament : file sections, easy logging, versions, history text .. robust - i know, we often want a "oomplete rewrite" :-) Let things sink in.

btw.here's a PDF of the lesson i gave yesterday at my club - theme "Zwischenzug" (in-between move) : 240513-8-puzzles.pdf

HTH !

@ understanding @ @ knowledge @ @ wisdom @

[tissatussa]

btw. lately i only use a binary compiled of PR4, after your TT fix. should i test with another version ? maybe it gives the same UCI output as you showed ..

          `-/osyhddddhyso/-`              roelof@roelof-HP-Elite-x2-1012-G2 
        .+yddddddddddddddddddy+.           --------------------------------- 
      :yddddddddddddddddddddddddy:         OS: Xubuntu 22.04.4 LTS x86_64 
    -yddddddddddddddddddddhdddddddy-       Host: HP Elite x2 1012 G2 
   odddddddddddyshdddddddh`dddd+ydddo      Kernel: 5.15.0-107-generic 
 `yddddddhshdd-   ydddddd+`ddh.:dddddy`    Uptime: 5 hours, 54 mins 
 sddddddy   /d.   :dddddd-:dy`-ddddddds    Packages: 3955 (dpkg), 19 (snap) 
:ddddddds    /+   .dddddd`yy`:ddddddddd:   Shell: bash 5.1.16 
sdddddddd`    .    .-:/+ssdyodddddddddds   Resolution: 1920x1080, 1920x1080 
ddddddddy                  `:ohddddddddd   DE: Xfce 4.16 
dddddddd.                      +dddddddd   WM: Xfwm4 
sddddddy                        ydddddds   WM Theme: Default 
:dddddd+                      .oddddddd:   Theme: Adwaita-dark [GTK2/3] 
 sdddddo                   ./ydddddddds    Icons: elementary-xfce-darker [GTK2/3] 
 `yddddd.              `:ohddddddddddy`    Terminal: xfce4-terminal 
   oddddh/`      `.:+shdddddddddddddo      Terminal Font: DejaVu Sans Mono 9 
    -ydddddhyssyhdddddddddddddddddy-       CPU: Intel i5-7200U (4) @ 3.100GHz 
      :yddddddddddddddddddddddddy:         GPU: Intel HD Graphics 620 
        .+yddddddddddddddddddy+.           Memory: 6249MiB / 7816MiB 
           `-/osyhddddhyso/-`                                                                

[tissatussa]

in the past i did experiments with EPD a set of 'puzzles', with their 'bm' and 'am' data and see the reports .. Best Move can be more than 1, also Avoid Move (not many puzzle EPDs exist with this move type) .. a4 would be an 'am' here.

[skycloudd]

btw. lately i only use a binary compiled of PR4, after your TT fix. should i test with another version ? maybe it gives the same UCI output as you showed ..

I've now merged that pr, so if you could use the latest version (6614a95) that'd be good :)

[skycloudd]

in the past i did experiments with EPD a set of 'puzzles', with their 'bm' and 'am' data and see the reports .. Best Move can be more than 1, also Avoid Move (not many puzzle EPDs exist with this move type) .. a4 would be an 'am' here.

I'll look into that! Sounds like a good way to test

[tissatussa]

i compiled that version and here are my 2 outputs of the position. both have almost identical data :

$ ./eccat-v0.1.2-PR4-fix2-commit-6614a95
eccat v0.1.2 (VERGEN_IDEMPOTENT_OUTPUT) by skycloudd
(VERGEN_IDEMPOTENT_OUTPUT, 2024-05-15) [Rust 1.78.0] on Ubuntu
uci
id name eccat v0.1.2 (VERGEN_IDEMPOTENT_OUTPUT)
id author skycloudd
option name Hash type spin default 16 min 1 max 33554432
uciok
ucinewgame
position fen 1r1r2k1/1bbp1p2/2p2p1p/p1q5/4PP2/PPN2B1P/1K1Q2P1/3RR3 w - - 1 23
go infinite
info depth 1 seldepth 6 time 0 score cp 52 nodes 105 nps 485731 hashfull 0 pv f4f5
info depth 2 seldepth 6 time 0 score cp 34 nodes 762 nps 917307 hashfull 0 pv a3a4 b8c8
info depth 3 seldepth 9 time 3 score cp 51 nodes 4742 nps 1446017 hashfull 0 pv a3a4 c5b6 f4f5
info depth 4 seldepth 11 time 6 score cp 37 nodes 8711 nps 1348832 hashfull 0 pv a3a4 f6f5 b2b1 c7f4 d2f4 c5c3 f4h6 f5e4 f3e4 c3b3
info depth 5 seldepth 12 time 16 score cp 48 nodes 24357 nps 1469714 hashfull 0 pv a3a4 c5b6 e4e5 f6e5 f4e5
info depth 6 seldepth 17 time 38 score cp 35 nodes 49243 nps 1271377 hashfull 2 pv a3a4 c7b6 b2b1 c5f2 d2d6 g8g7
info depth 7 seldepth 17 time 114 score cp 39 nodes 142509 nps 1241681 hashfull 5 pv h3h4 c5e7 f3g4 d7d6 f4f5 g8g7 b2b1
info depth 8 seldepth 18 time 465 score cp 39 nodes 515400 nps 1107993 hashfull 20 pv h3h4 c5e7
info depth 9 seldepth 22 time 2842 score cp 35 nodes 3390782 nps 1192933 hashfull 138 pv d1c1 c7b6 f4f5 c5f2 e1e2 f2h4 c1d1 b8c8 b2b1
info depth 10 seldepth 27 time 16101 score cp 25 nodes 19202057 nps 1192540 hashfull 665 pv c3a4 c5e7 g2g3 d7d6 a4c3 d8e8 d2c2 b8d8 b2b1 c7b6
info depth 11 seldepth 27 time 25034 score cp 24 nodes 42970654 nps 1716476 hashfull 956 pv c3a4 c5f8 a4c3 f8g7 g2g4 b8c8 b2b1 d7d6 f4f5 g7g5 d2f2
info depth 12 seldepth 27 time 41234 score cp 24 nodes 83006000 nps 2013016 hashfull 999 pv c3a4 c5e7 b2a2 b7a6 d2e3 a6b7 f3h5 g8h8 a4b6 d7d5 h5f3 c7b6 e3b6 d5e4 e1e4
info depth 13 seldepth 28 time 62198 score cp 26 nodes 135045943 nps 2171217 hashfull 1000 pv d1c1 c7b6 e1d1 c5f2 d2f2 b6f2 d1d6 g8g7 c1d1 f2g3 a3a4 g3f4 d6d7 d8d7 d1d7
info depth 14 seldepth 31 time 114223 score cp 24 nodes 230937554 nps 2021798 hashfull 1000 pv c3a4 c5e7 g2g3 d7d6 f3g2 b7a6 d2c2 d8c8 b2c1 e7e6 e1e3 a6b7 a4c3 d6d5
info depth 15 seldepth 34 time 249350 score cp 27 nodes 531080936 nps 2129858 hashfull 1000 pv d1c1 c5d6 d2d6 c7d6 e4e5 f6e5 c3e4 d6c7 e4f6 g8g7 f6h5 g7g6 f4e5 c7b6 c1d1 b6f2 d1d6 g6h7

within 1 seconds the move a4 is no longer considered, so this version seems OK !

[tissatussa]

btw. i did set Hash 16 Mb, your default.

[skycloudd]

eccat now no longer chooses a3a4 from depth 9 and later (which it reaches in ~400ms on my laptop, with two new optimisations ive added) so this is no longer an issue :D thank you!