search

niklasf / python-chess

A chess library for Python, with move generation and validation, PGN parsing and writing, Polyglot opening book reading, Gaviota tablebase probing, Syzygy tablebase probing, and UCI/XBoard engine communication
https://python-chess.readthedocs.io/en/latest/
GNU General Public License v3.0
2.45k stars 531 forks source link

Using limit(mate=mate) terminates search before mate #1070

Closed robertnurnberg closed 8 months ago

robertnurnberg commented 8 months ago

Running the attached script several times will eventually yield an output of the form below.

> python python-chess-test.py
score #+1 (d1, nodes 35) PV: d1a1
score -884 (d4, nodes 109) PV: b4c5 d2d4
score +980 (d6, nodes 1912) PV: d3b3
score -951 (d5, nodes 768) PV: c5b5 d3b3
score #+3 (d16, nodes 203487) PV: b1d1 c5b6 d3b3 b6c5 d2d4
score #-3 (d24, nodes 36341) PV: h4h3 b1d1 c5b6 d3b3 b6a5 d1a1
score #+4 (d25, nodes 135315) PV: b8b1 h4h3 b1d1 c5b6 d3b3 b6a7 d1a1

Notice the line score +980 (d6, nodes 1912) PV: d3b3 which should really be a score #+2.

The output also highlights an issue with mated-in positions, where python-chess will regularly stop the search before the requested mate is reached.

The script I used is as follows.

import chess, chess.engine

engine = "./stockfish"

l = [
    ("8/3p4/2pP4/4P3/k7/1R2N2p/2PP3K/3R4 w - -", 1),
    ("8/3p4/2pP4/4P3/1k6/1R2N2p/2PP3K/3R4 b - -", 1),
    ("8/3p4/2pP4/4P3/1k6/3RN2p/2PP3K/3R4 w - -", 2),
    ("8/3p4/2pP4/2k1P3/8/3RN2p/2PP3K/3R4 b - -", 2),
    ("8/3p4/2pP4/2k1P3/8/3RN2p/2PP3K/1R6 w - -", 3),
    ("8/3p4/2pP4/2k1P3/7p/3RN3/2PP3K/1R6 b - -", 3),
    ("1R6/3p4/2pP4/2k1P3/7p/3RN3/2PP3K/8 w - -", 4),
]

with chess.engine.SimpleEngine.popen_uci(engine) as engine:
    engine.configure({"Threads": 6})
    for fen, mate in l:
        board = chess.Board(fen)
        info = engine.analyse(board, chess.engine.Limit(mate=mate), game=board)
        if "score" in info:
            score = info["score"].pov(board.turn)
            depth = info["depth"] if "depth" in info else None
            nodes = info["nodes"] if "nodes" in info else None
            pv = [m.uci() for m in info["pv"]] if "pv" in info else []
            print(f"score {score} (d{depth}, nodes {nodes}) PV: {' '.join(pv)}")
niklasf commented 8 months ago

This appears to be a bug in Stockfish, including the latest 16.1.

UCI input:

setoption name Threads value 6
position fen 8/3p4/2pP4/4P3/1k6/3RN2p/2PP3K/3R4 w - - 0 1
go mate 2

Occasionally observing output like:

Stockfish 16.1 by the Stockfish developers (see AUTHORS file)
info string NNUE evaluation using nn-baff1ede1f90.nnue
info string NNUE evaluation using nn-b1a57edbea57.nnue
info depth 1 seldepth 4 multipv 1 score cp 769 nodes 996 nps 996000 hashfull 0 tbhits 0 time 1 pv d3b3 b4a4
info depth 2 seldepth 4 multipv 1 score cp 814 nodes 1667 nps 1667000 hashfull 0 tbhits 0 time 1 pv d3b3 b4a5 h2h3
info depth 4 seldepth 4 multipv 1 score cp 960 nodes 1667 nps 1667000 hashfull 0 tbhits 0 time 1 pv d3b3
bestmove d3b3 ponder b4a5
robertnurnberg commented 8 months ago

Ouch, thanks for spotting this.

robertnurnberg commented 8 months ago

Ok, so it seems in some cases stockfish does not send the last uci info with the mate score. Closing this issue.