search

official-stockfish / Stockfish

A free and strong UCI chess engine
https://stockfishchess.org/
GNU General Public License v3.0
11.72k stars 2.3k forks source link

getting mate 0 in MultiPV mode, but position is not mate #502

Closed PascalPflaum closed 7 years ago

PascalPflaum commented 9 years ago

I noticed this problem with Stockfish 6 64 POPCNT.

Here the essential communication over uci.

Mon Nov 16 2015 12:42:01 GMT+0100 (CET) >>>> uci Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< Stockfish 6 64 POPCNT by Tord Romstad, Marco Costalba and Joona Kiiski Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< id name Stockfish 6 64 POPCNT Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< id author Tord Romstad, Marco Costalba and Joona Kiiski Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Write Debug Log type check default false Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Contempt type spin default 0 min -100 max 100 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Min Split Depth type spin default 0 min 0 max 12 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Threads type spin default 1 min 1 max 128 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Hash type spin default 16 min 1 max 1048576 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Clear Hash type button Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Ponder type check default true Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name MultiPV type spin default 1 min 1 max 500 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Skill Level type spin default 20 min 0 max 20 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Move Overhead type spin default 30 min 0 max 5000 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Minimum Thinking Time type spin default 20 min 0 max 5000 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Slow Mover type spin default 80 min 10 max 1000 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name UCI_Chess960 type check default false Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name SyzygyPath type string default Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name SyzygyProbeDepth type spin default 1 min 1 max 100 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name Syzygy50MoveRule type check default true Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< option name SyzygyProbeLimit type spin default 6 min 0 max 6 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< uciok Mon Nov 16 2015 12:42:01 GMT+0100 (CET) >>>> isready Mon Nov 16 2015 12:42:01 GMT+0100 (CET) <<<< readyok Mon Nov 16 2015 12:42:01 GMT+0100 (CET) >>>> setoption name MultiPV value 3 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) >>>> setoption name Hash value 32 Mon Nov 16 2015 12:42:01 GMT+0100 (CET) >>>> setoption name Threads value 1 Mon Nov 16 2015 12:42:30 GMT+0100 (CET) >>>> ucinewgame Mon Nov 16 2015 12:42:30 GMT+0100 (CET) >>>> position fen 5r2/4q1k1/3p2p1/2pPp1bp/1pP1P1N1/rP4PQ/5P2/1R1R2K1 w - - 0 1 Mon Nov 16 2015 12:42:30 GMT+0100 (CET) >>>> setoption name MultiPV value 3 Mon Nov 16 2015 12:42:30 GMT+0100 (CET) >>>> go nodes 30000000

and then after some time

Mon Nov 16 2015 12:42:38 GMT+0100 (CET) <<<< info depth 19 seldepth 35 multipv 1 score cp -187 nodes 5713122 nps 755604 tbhits 0 time 7561 pv g4h2 a3a2 d1f1 a2e2 b1d1 e2e4 d1e1 e4e1 f1e1 e7f7 h3g2 g5d2 e1f1 d2c3 g2e4 f7f5 e4f5 f8f5 f2f3 c3d4 g1g2 f5f8 f1e1 f8a8 h2f1 a8a3 Mon Nov 16 2015 12:42:38 GMT+0100 (CET) <<<< info depth 19 seldepth 35 multipv 2 score cp -437 nodes 5713122 nps 755604 tbhits 0 time 7561 pv g4e3 g5e3 f2e3 e7g5 g1h1 g5e3 d1e1 e3f3 h3g2 a3b3 g2f3 f8f3 b1b3 f3b3 h1g2 b3c3 e1a1 c3c4 g2f3 c4c3 f3f2 c3a3 a1b1 g7f6 Mon Nov 16 2015 12:42:38 GMT+0100 (CET) <<<< info depth 18 seldepth 35 multipv 3 score mate 0 nodes 5713122 nps 755604 tbhits 0 time 7561 pv d1f1 e7d7 Mon Nov 16 2015 12:42:38 GMT+0100 (CET) >>>> stop Mon Nov 16 2015 12:42:38 GMT+0100 (CET) <<<< bestmove g4h2 ponder a3a2

the score mate 0 in the multipv 3 is wrong

stockfishMate0.txt

I attached a more verbose log file.

zamar commented 8 years ago

This is still an open issue, so I prefer to keep this open.

joergoster commented 8 years ago

@zamar I don't think this can be solved in such a way, that this never happens at all. We might increase the chance that 'bool updated' will be true more often with

      bool updated = (i <= PVIdx || rootMoves[i].score != -VALUE_INFINITE);

but this doesn't eliminate the problem. If the search is stopped in an unlucky moment, updated may still be false and the previous score (-VALUE_INFINITE) will be printed.

joergoster commented 8 years ago

In addition, here some numbers for dbg_hit_on(updated == true).

./stockfish uci setoption name MultiPV value 6 position fen 5r2/4q1k1/3p2p1/2pPp1bp/1pP1P1N1/rP4PQ/5P2/1R1R2K1 w - - 0 1 go movetime 10000

master

bool updated = (i <= PVIdx);
Total 198 Hits 158 hit rate (%) 79

with my proposed addition

bool updated = (i <= PVIdx || rootMoves[i].score != -VALUE_INFINITE);
Total 198 Hits 163 hit rate (%) 82

with the 'fix' proposed on fishcooking forum

bool updated = (i <= PVIdx && rootMoves[i].score != -VALUE_INFINITE);
Total 198 Hits 158 hit rate (%) 79
joergoster commented 8 years ago

@zamar @mcostalba Maybe we should simply print "no score " in this case? Edit: Ok, simply ignore. UCI protocol demands either cp or mate string.

MichaelB7 commented 8 years ago

I can no longer reproduce this behavior in current Stockfish. Does anyone else want to try?

Rocky640 commented 8 years ago

@MichaelB7 : are you on windows or linux ? and are you able to reproduce the original problem (on Stockfish 6 64 POPCNT)

joergoster commented 7 years ago

It still can happen if the search gets stopped in an unfortunate moment.

info depth 23 seldepth 30 multipv 1 score cp 0 nodes 1025625496 nps 13844836 hashfull 653 tbhits 6660393 time 74080 pv h5g5 e8d7 a2f7 d7c8 f7d5 c8d7 d5a2 d7c8 a2f7 c8d7 f7a2
info depth 23 seldepth 30 multipv 2 score mate 0 upperbound nodes 1025625496 nps 13844836 hashfull 653 tbhits 6660393 time 74080 pv h5h6 e8a4 a2f7 a4b3 f7e8 c3c4 e8f7 c4c3
joergoster commented 7 years ago

I have pushed a possible fix here: https://github.com/joergoster/Stockfish/tree/issue502

In general, this patch suppresses output of the new PVIdx if we don't have a valid score. This can happen if the search has been stopped in an unfortunate moment while still in the aspiration loop. We have to address two cases. Case 1: The new PVIdx was already part of the k-best pv's in the last iteration, and we therefore have a valid pv and score to output from the last iteration. This is taken care of with:

 +      bool updated = (i <= PVIdx && rootMoves[i].score != -VALUE_INFINITE);

Case 2: The new PVIdx was NOT part of the k-best pv's in the last iteration, and we have no valid pv and score to output. Not from the current nor from the previous iteration. In this case the above patch is not sufficient, and following addition is taking care of this:

+      if (   !updated
+          && (depth == ONE_PLY || rootMoves[i].previousScore == -VALUE_INFINITE))
           continue;

I also added an assertion in UCI::value() to possibly catch such bugs earlier. https://github.com/joergoster/Stockfish/commit/2f5b10a66df99913b0d48a6be7a573abe6decfed (I consider this a bug because the range of valid scores is from -VALUE_MATE to VALUE_MATE.)

So far, I am not getting mate 0 outputs any longer with this fix. As always, it would be good if other people could confirm. Thanks.