fairy-stockfish / Fairy-Stockfish

chess variant engine supporting Xiangqi, Shogi, Janggi, Makruk, S-Chess, Crazyhouse, Bughouse, and many more
https://fairy-stockfish.github.io/
GNU General Public License v3.0
606 stars 189 forks source link

Reporting a draw as a win #824

Open tolius opened 2 weeks ago

tolius commented 2 weeks ago

fairy-stockfish_x86-64-bmi2.exe (Fairy-Stockfish 14 BMI2, 8x8 variants, Windows):

setoption name UCI_Variant value antichess
position fen 7N/8/B3P3/7p/5P2/8/1P2P3/1NBQK1N1 w - - 0 22
go depth 18

Result:

info depth 18 seldepth 29 multipv 1 score mate 14 nodes 34086799 nps 889367 hashfull 1000 tbhits 0 time 38327 pv e6e7 h5h4 g1f3 h4h3 f3g1 h3h2 e2e4 h2g1b e1f2 g1f2 c1e3 f2e3 e4e5 e3f4 a6b7 f4e5 d1a4 e5h8 e7e8q h8b2 b1c3
bestmove e6e7 ponder h5h4

The engine thinks it can win in this drawn position in 14 moves, yet offers a sequence of moves that leads to a draw.

ianfab commented 1 week ago

Thanks for reporting. By some weird coincidence exactly the most recent commit I pushed two weeks ago seems to fix or at least work around this particular issue. Before https://github.com/fairy-stockfish/Fairy-Stockfish/commit/f474f12af3786a98393227e0b4b335434c5b8880 I can still reproduce this exact behavior even with recent non-largeboard versions. However, my hunch is that this more likely only hides than really fixes whatever causes this bug. One potential cause could be a static eval overflowing into the mate range, but for now that is just a shot in the dark.

ianfab commented 1 week ago

After having had a bit of a look at it, to me it looks like this might be a hash collision. When I change the hash size or verify more bits of the hash (32 instead of 16) in the TT the issue does not appear any more. It would be good to narrow it down further, but for now I assume that there likely isn't a direct fix for it and it might just be an unlucky case.

tolius commented 1 week ago

With the latest changes, I was able to reproduce the same thing in a slightly different position:

setoption name UCI_Variant value antichess
position fen 7B/8/B3P3/8/5P1p/8/1P2P3/1NBQK1N1 w - - 0 22
go depth 27

It announces a win somewhere at depth 21. None of the asserts triggered.

It is much easier to reproduce in this position, which cannot occur in a real game:

setoption name UCI_Variant value antichess
position fen 8/3P1P1P/B1P1P1P1/3P1P1P/2P1P1P1/3P1P1P/2P1P1Pb/8 w - - 0 1
go depth 10

where this assert is triggered almost instantly:

Assertion failed: eg_value(result) == (i * eg_value(s))

types.h, line 698 But this is probably expected because of too many pawns.