Disservin / fastchess

fastchess is a chess cli tool to run engine vs engine matches
MIT License
84 stars 20 forks source link

question about illegal bestmove handling #377

Closed gahtan-syarif closed 5 months ago

gahtan-syarif commented 5 months ago

how does fastchess currently handle engines throwing illegal bestmoves? cuz i compared cutechess and fastchess using a purposefully broken version of stormphrax, and this is what i found: when cutechess detects an illegal bestmove it automatically adjudicates the game and gives the loss to the engine that displays the illegal bestmove, as shown here:

Warning: Illegal PV move g2b1 from Engine2 (1)
Warning: PV: Rh6 Kg5 Rh2 Re7+ g2b1
Warning: Illegal PV move e6b4 from Engine1 (0)
Warning: PV:  e6b4
Finished game 1 (Engine1 vs Engine2): 1-0 {White wins by adjudication}
Score of Engine1 vs Engine2: 1 - 0 - 0  [1.000] 1
...      Engine1 playing White: 1 - 0 - 0  [1.000] 1
...      White vs Black: 1 - 0 - 0  [1.000] 1
Elo difference: inf +/- nan, LOS: 84.1 %, DrawRatio: 0.0 %
SPRT: llr 0 (0.0%), lbound -inf, ubound inf

while in fastchess, the program just ended suddenly, im not sure if its a crash or the program just exiting:

Warning; Illegal pv move  b2b4 pv: info depth 21 seldepth 43 time 480 nodes 853014 nps 1773944 score cp -50 wdl 2 877 121 hashfull 9 pv b2c1r d1d2 a8b8 b2b4 a5b4 c3b4 c6c5 b4c3 b6b5 c4a5 e7e8 f1b1 b5b4 c3b2 b8a8 a5c4 e8e6 a2a3 c8a6 c4e3 a6d3 d2d3 f8b8 a3b4 a8a1
Warning; Illegal pv move  c3b4 pv: info depth 21 seldepth 43 time 480 nodes 853014 nps 1773944 score cp -50 wdl 2 877 121 hashfull 9 pv b2c1r d1d2 a8b8 b2b4 a5b4 c3b4 c6c5 b4c3 b6b5 c4a5 e7e8 f1b1 b5b4 c3b2 b8a8 a5c4 e8e6 a2a3 c8a6 c4e3 a6d3 d2d3 f8b8 a3b4 a8a1
Warning; Illegal pv move  e7e8 pv: info depth 21 seldepth 43 time 480 nodes 853014 nps 1773944 score cp -50 wdl 2 877 121 hashfull 9 pv b2c1r d1d2 a8b8 b2b4 a5b4 c3b4 c6c5 b4c3 b6b5 c4a5 e7e8 f1b1 b5b4 c3b2 b8a8 a5c4 e8e6 a2a3 c8a6 c4e3 a6d3 d2d3 f8b8 a3b4 a8a1
Warning; Illegal pv move  f1b1 pv: info depth 21 seldepth 43 time 480 nodes 853014 nps 1773944 score cp -50 wdl 2 877 121 hashfull 9 pv b2c1r d1d2 a8b8 b2b4 a5b4 c3b4 c6c5 b4c3 b6b5 c4a5 e7e8 f1b1 b5b4 c3b2 b8a8 a5c4 e8e6 a2a3 c8a6 c4e3 a6d3 d2d3 f8b8 a3b4 a8a1

C:\Users\Gahtan Syarif Nahdi\Downloads\fast-chess-fixratinginterval\fast-chess-fixratinginterval>

is this a bug or a design feature? if its a design feature can we make it so that its consistent with cutechess?

Disservin commented 5 months ago

Ah the error is here https://github.com/Disservin/fast-chess/blob/master/src/matchmaking/match/match.cpp#L287 the loop needs to stop when an illegal pv move is found

gahtan-syarif commented 5 months ago

ahh so its a bug then

gahtan-syarif commented 5 months ago

Stormphrax-random_illegals.zip btw this is the broken sp source code if you wanna test things out yourself

gahtan-syarif commented 5 months ago

fixed with https://github.com/Disservin/fast-chess/commit/a37e9d90ef70cd639d00d4fa52d444879c51a34c