search

niklasf / python-chess

A chess library for Python, with move generation and validation, PGN parsing and writing, Polyglot opening book reading, Gaviota tablebase probing, Syzygy tablebase probing, and UCI/XBoard engine communication
https://python-chess.readthedocs.io/en/latest/
GNU General Public License v3.0
2.44k stars 530 forks source link

Deadlock in chess.uci #320

Closed niklasf closed 5 years ago

niklasf commented 5 years ago

Hi Niklas

The code is running on Ubuntu 16.04 x64 with python 3.6.6 as part of an anaconda environment the process that is hanging and taking up cpu resources is python and not the lc0 engine

I just found out that the problem manifests itself only after I run to a breakpoint (using the integrated python debugger of vscode). if I single step through the program, everything is fine. needless to say this has never happened before with other python libraries in the same environment

best regards

[...]

On Thu, Nov 1, 2018 at 9:41 PM Niklas Fiekas < niklas.fiekas@backscattering.de> wrote:

Hi [...],

the code I see here looks fine. Ultimately I might need a self contained example to reproduce this.

Some information that might help:

  • Python version and operating system
  • Is multi-threading involved?
  • Does the deadlock also happen when using another engine, e.g. Stockfish?
  • Which process is using CPU during the deadlock? (Python, LC0, ...)

Best, Niklas

On 01.11.18 00:04, [...] wrote:

Hi Niklas

one more question I have a strange bug when controlling the lc0 engine with the python-chess uci interface I have written a simple wrapper class so that I can keep the engine "alive", and send move prediction requests for certain board positions this is the code:

classLC0UciEngine(object):
""" UCI engine wrapper for lc0 """

def__init__(self, lc0_exec: str=_LC0, pv_num: int=8)->None:
self.lc0_exec =lc0_exec
self.pv_num =pv_num
self.engine =chess.uci.popen_engine(lc0_exec)
self.engine.uci()
self.engine.isready()
self.engine.setoption({'MultiPV': pv_num})
self.info_handler =chess.uci.InfoHandler()
self.engine.info_handlers.append(self.info_handler)

@staticmethod
deffen_to_board(fen: str)->Board:
board =Board()
board.set_fen(fen)
returnboard

defpredict_move(self, board_position: Board, node_limit: int)->Tuple:
self.engine.ucinewgame()
self.engine.position(board_position)
res =self.engine.go(nodes=node_limit)
returnres

defget_pv(self)->Dict:
pvs =self.info_handler.info <http://info_handler.info>['pv']
scores =self.info_handler.info <http://info_handler.info>['score']
result =[]
fori inrange(1, self.pv_num+1):
result +=[{'move': pvs[i][0], 'score':scores[i]}]
returnresult

when I run predict_move directly after constructing the wrapper class it works flawlessly, however if the calling program iterates through a few pgn games (using chess.pgn from your library), then the predict_move no longer works. I have verified that the self.engine variable still has the same address Also when adding the logging option you previously suggested I can see the following output of the wrapper initializer:

DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >>
option name Length of history to include in cache type spin default 0
min 0 max 7
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >>
option name Policy softmax temperature type string default 2.200000
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >>
option name Allowed node collisions, per batch type spin default 32 min
0 max 1024
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >>
option name Out-of-order cache backpropagation type check default true
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >>
option name MultiPV type spin default 1 min 1 max 500
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >>
option name Configuration file path type string default lc0.config
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >>
option name Do debug logging into file type string default
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >> uciok
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> <<
isready
Found pb network file:

/tmp/.local/bin/72941e34e6fedb21582e00434ca1efa1b2757176dbf47087b85415103034e90a
Creating backend [cudnn]...
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >>
readyok
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> <<
setoption name MultiPV value 8
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> <<
isready
DEBUG:chess.engine:<PopenProcess at 0x7f90ec054e48 (pid=31652)> >>
readyok

but issuing the 'ucinewgame()' command at the beginning of the predict_move method does not cause anything to be logged, the system halts with cpu % going up to 100%

any clue how I can figure it out?

best regards

[...]

On Wed, Oct 31, 2018 at 6:51 PM [...] wrote:

Hi Niklas

I have compiled a newer version of lc0 and it seems to work fine

thank you so much

[...]

On Wed, Oct 31, 2018 at 6:09 PM Niklas Fiekas
<niklas.fiekas@backscattering.de
<mailto:niklas.fiekas@backscattering.de>> wrote:

    Hi [...],

    please add

    ```
    import logging
    logging.basicConfig(level=logging.DEBUG)
    ```

    at the start of your program and try the same thing again. The
    additional debug output might help track down the problem.

    Best regards,
    Niklas

    On 31.10.18 16:52, [...] wrote:
    > hi
    >
    > I am using your python chess library, and I tried to control

the

leela-zero chess engine (lc0) using the uci interface. I have tested the lc0 executable by running it from the command line, and its uci interface seems to operate correctly. I can successfully connect using the chess.uci.popen_engine command, but I get the following exceptions when issuing the uci command (engine.uci()):

>>> engine.uci()
exception parsing option max
Traceback (most recent call last):
  File
"/home/user/anaconda3/lib/python3.6/site-packages/chess/uci.py",
line 634, in _option
    max = int(token)
ValueError: invalid literal for int() with base 10: 'nodes'
exception parsing option max
Traceback (most recent call last):
  File
"/home/user/anaconda3/lib/python3.6/site-packages/chess/uci.py",
line 634, in _option
    max = int(token)
ValueError: invalid literal for int() with base 10: 'to'
exception parsing option max
Traceback (most recent call last):
  File
"/home/user/anaconda3/lib/python3.6/site-packages/chess/uci.py",
line 634, in _option
    max = int(token)
ValueError: invalid literal for int() with base 10: 'use'
exception parsing option max
Traceback (most recent call last):
  File
"/home/user/anaconda3/lib/python3.6/site-packages/chess/uci.py",
line 634, in _option
    max = int(token)
ValueError: invalid literal for int() with base 10: 'time'
exception parsing option max
Traceback (most recent call last):
  File
"/home/user/anaconda3/lib/python3.6/site-packages/chess/uci.py",
line 634, in _option
    max = int(token)
ValueError: invalid literal for int() with base 10: 'to'
exception parsing option max
Traceback (most recent call last):
  File
"/home/user/anaconda3/lib/python3.6/site-packages/chess/uci.py",
line 634, in _option
    max = int(token)
ValueError: invalid literal for int() with base 10: 'us'
exception parsing option max
Traceback (most recent call last):
  File
"/home/user/anaconda3/lib/python3.6/site-packages/chess/uci.py",
line 634, in _option
    max = int(token)
ValueError: invalid literal for int() with base 10: 'in'
exception parsing option max
Traceback (most recent call last):
  File
"/home/user/anaconda3/lib/python3.6/site-packages/chess/uci.py",
line 634, in _option
    max = int(token)
ValueError: invalid literal for int() with base 10: 'ms'
>>>

I am guessing this has to do with the following:

|options|

A case-insensitive dictionary of Options

<https://python-chess.readthedocs.io/en/latest/uci.html#options . The engine should send available options when it receives the initial /uci/ command.

I know that lc0 has been successfully integrated as a uci engine in various chess gui programs. I would appreciate your help in figuring out what is wrong and how to get python-chess to control lc0

thanks

[...]

QueensGambit commented 5 years ago

Hey, one thing you could try out is calling the python interface of leela using a .sh file in linux. It took me some time to figure this out, but it worked on my setup for CrazyAra:

Use a .sh-file to start python from a different process:

crazyara.sh

#!/bin/sh
/home/queensgambit/anaconda3/bin/python /path/crazyara.py

test_engine_loading.py

import chess.uci
engine = chess.uci.popen_engine('/path/crazyara.sh') 
engine.uci()
engine.name
# >> 'CrazyAra 0.3.0'
engine.isready()
engine.go()
# >> BestMove(bestmove=Move.from_uci('e2e4'), ponder=None)

Best regards, ~QueensGambit