interact with chess binary

[tissatussa]

i found your BU Nim package, which might be useful for my coding idea : a terminal script to communicate with 'stockfish', a well-known chess engine binary, using a thread (or 2) .. i tried several solutions, but i can't get it to work : just give a UCI (protocol) command by stdin and catch its output by stdout.

i'd not expect this to be hard .. can you supply a basic script for this ? Or point me to some function / example ?

i use Nim 2.0.6 i'm on Xubuntu 22.04

[c-blake]

I am really unsure how much I can help you. I am a little familiar with chess but haven't played in decades. I'm not very familiar with UCI. What you describe seems more like a text protocol with simple parsing & printing (& whatever else you are doing on the outside of your stockfish). So, I agree this should not be hard, and Nim does make things like parsing / printing simple text things easy, just with the Nim stdlib.

However, what kind of data structure / type you want to load from stdin & print to stdout (or maybe vice-versa) probably depends on the UI or analysis or whatever you are trying to build and you haven't said much about that. This bu package in particular is also a bunch of very small CLI tools and I cannot really imagine. So, this is not really the best place to discuss this or general Nim programming. I am also traveling right now. So, my ability to reply in detail is limited. The basic stub of your logic might be something like:

import std/syncio
type
  Extra = enum Castle # , etc., etc.
  Coord = tuple[x,y: int]
  Move = tuple[frm, to: Coord; aux: Extra]
proc parseCoord(mv: var Coord; s: string, start=0): bool =
  discard # fill me in
var chMv: Move
for line in lines stdin:
  if not parseCoord(chMv.frm, line):
    stderr.write "bad line: ", line, "\n"
  if not parseCoord(chMv.to, line, 2):
    stderr.write "bad line: ", line, "\n"
  #... parseExtra etc., etc.
  # At some point the line is fully parsed and maybe do something else

Since this is not really some kind of "Unix" or "statistical data analysis" or other tool of broader generality than the game of Chess, I am going to close this issue. But you can ask more questions if you like.

[tissatussa]

i will give you the basics of running a chess engine binary (on linux here) with the UCI protocol : giving commands by stdin and catch its output by stdout.

Info UCI : see https://www.chessprogramming.org/UCI and https://github.com/fsmosca/UCIChessEngineProtocol

UCI has many features & functions, but in practice it simply goes like this (i use the open source stockfish (SF) engine, stable and very strong), see terminal display below.

first set usage of the UCI protocol by 'uci' : SF returns its '@@@' info lines (my indication) and these should end with 'uciok'. then 'ucinewgame' initiates a new game, no response will be given. then we set the starting position, no response will be given. then we let the engine run with some max-time-per-game (w/b=white/black / 'inc'=time-INCrement-after-each-move) and it shows its continues output until the line 'bestmove [..]' is given.

that's all, this type of command sequence repeats for every move : the next position is derived by a command like 'position startpos move1 move2 move3 ...' - so, until the current position is reached.

---

$ stockfish
@@@ Stockfish 14.1 by the Stockfish developers (see AUTHORS file)
uci
@@@ id name Stockfish 14.1
@@@ id author the Stockfish developers (see AUTHORS file)
@@@ option name Debug Log File type string default
@@@ option name Threads type spin default 1 min 1 max 512
@@@ option name Hash type spin default 16 min 1 max 33554432
@@@ option name Clear Hash type button
@@@ option name Ponder type check default false
@@@ option name MultiPV type spin default 1 min 1 max 500
@@@ option name Skill Level type spin default 20 min 0 max 20
@@@ option name Move Overhead type spin default 10 min 0 max 5000
@@@ option name Slow Mover type spin default 100 min 10 max 1000
@@@ option name nodestime type spin default 0 min 0 max 10000
@@@ option name UCI_Chess960 type check default false
@@@ option name UCI_AnalyseMode type check default false
@@@ option name UCI_LimitStrength type check default false
@@@ option name UCI_Elo type spin default 1350 min 1350 max 2850
@@@ option name UCI_ShowWDL type check default false
@@@ option name SyzygyPath type string default <empty>
@@@ option name SyzygyProbeDepth type spin default 1 min 1 max 100
@@@ option name Syzygy50MoveRule type check default true
@@@ option name SyzygyProbeLimit type spin default 7 min 0 max 7
@@@ option name Use NNUE type check default true
@@@ option name EvalFile type string default nn-13406b1dcbe0.nnue
@@@ uciok

ucinewgame position startpos go wtime 70000 btime 70000 winc 10000 binc 10000

@@@ info string NNUE evaluation using nn-13406b1dcbe0.nnue enabled
@@@ info depth 1 seldepth 1 multipv 1 score cp 38 nodes 20 nps 10000 tbhits 0 time 2 pv d2d4
@@@ (--more--)
@@@ info depth 18 seldepth 21 multipv 1 score cp 39 nodes 552434 nps 525127 hashfull 232 tbhits 0 time 1052 pv d2d4 g8f6 g1f3 d7d5 c2c4 e7e6 b1c3 f8e7 c1g5 b8d7 e2e3 e8g8 a1c1 c7c6 d1c2 f6e4 g5e7 d8e7
... sinfo depth 19 seldepth 19 multipv 1 score cp 39 nodes 588356 nps 530050 hashfull 240 tbhits 0 time 1110 pv d2d4 g8f6 g1f3 d7d5 c2c4 e7e6 b1c3 f8e7 c1g5 b8d7 e2e3 e8g8 a1c1 c7c6 d1c2 f6e4 g5e7 d8e7 c4d5
... tinfo depth 20 seldepth 27 multipv 1 score cp 46 nodes 924634 nps 530180 hashfull 376 tbhits 0 time 1744 pv d2d4 d7d5 c2c4 e7e6 b1c3 g8f6 c1g5 f8b4 g1f3 h7h6 g5f6 d8f6 d1a4 b8c6 e2e3 e8g8 a1c1 b4c3 c1c3 e6e5
... opinfo depth 21 seldepth 27 multipv 1 score cp 45 nodes 1042162 nps 532258 hashfull 417 tbhits 0 time 1958 pv d2d4 d7d5 c2c4 e7e6 b1c3 g8f6 c1g5 f8e7 e2e3 e8g8 g1f3 b8d7 a1c1 h7h6 g5h4 c7c5 c4d5 f6d5 h4e7 d5e7 d4c5
@@@ info depth 22 seldepth 27 multipv 1 score cp 45 nodes 1338001 nps 532643 hashfull 519 tbhits 0 time 2512 pv d2d4
@@@ bestmove d2d4 ponder d7d5

go wtime 70000 btime 70000 winc 10000 binc 10000

@@@ info string NNUE evaluation using nn-13406b1dcbe0.nnue enabled
@@@ info depth 1 seldepth 1 multipv 1 score cp 45 nodes 31 nps 15500 tbhits 0 time 2 pv d2d4
@@@ (--more--)
@@@ info depth 23 seldepth 27 multipv 1 score cp 39 nodes 552434 nps 525127 hashfull 232 tbhits 0 time 1052 pv d2d4 g8f6 g1f3 d7d5 c2c4 e7e6 b1c3 f8e7 c1g5 b8d7 e2e3 e8g8 a1c1 c7c6 d1c2 f6e4 g5e7 d8e7
stop
@@@ info depth 24 seldepth 29 multipv 1 score cp 44 nodes 1920301 nps 513724 hashfull 714 tbhits 0 time 3738 pv e2e4
@@@ bestmove e2e4 ponder e7e5

---

in the above listing you can see me entering the command 'stop', best shown in the second run : i paste 'stop[enter]' in terminal and its feeded into stdin -- Note : the first run shows me entering 's','t','op' (the '...' lines) but then SF also gets the 'stop' command by stdin. Btw. SF is our reference, its logic and code are great.

('ponder' means the expected opponent reaction move)

creating my script idea in Python is not too hard, many examples exist, and even complete UCI modules / APIs .. but first i wanted to create a Nim GUI for the SF process (expecting the communication itself to be easy with threads), using the Owlkettle Widget builder for Nim : i opened an Issue there but my idea gave trouble along the way for 'the experts', pointing to some bug or short-coming of their Owlkettle or Nim or both .. also depending on versions, i don't remember exactly, but it isn't solved yet .. so, i might eventually use another "Widget kit" to build a GUI for it .. here's the Issue :

Text View widget : thread process can not change text buffer https://github.com/can-lehmann/owlkettle/issues/145

i also had contact with Stefan Salewski, he's the one who wrote a very nice PDF about Nim programming, but lately he changed his mind (due to some bad experience in the Nim community) and turned to Rust, so he couldn't help me either !?

last thing for now : here are 2 "code pieces" i composed, which don't work, but they can give you the idea, i think .. honestly, ChatGPT helped here, but i'm stuck, being fully aware of this AI short-comings ..

---

import osproc, strutils, streams, os

# Function to run the Stockfish engine as a separate process
proc runStockfish() =
  let stockfishCmd = "stockfish"  # Assuming Stockfish is in the PATH
  var stockfishProcess = startProcess(stockfishCmd, options={poUsePath, poStdErrToStdOut})

  defer:
    close(stockfishProcess)

  # Send the initial UCI command
  stockfishProcess.input.writeLine("uci")
  stockfishProcess.input.flush()

  # Main loop to handle user input and engine output
  while true:
    if stdin.ready():  # Check if the user entered a command
      let userCommand = stdin.readLine().strip()
      if userCommand == "quit":
        stockfishProcess.input.writeLine("quit")
        stockfishProcess.input.flush()
        break
      stockfishProcess.input.writeLine(userCommand)
      stockfishProcess.input.flush()

    # Check and display the engine's output
    while stockfishProcess.output.peek() > 0:
      let engineOutput = stockfishProcess.output.readLine().strip()
      if engineOutput.len > 0:
        echo "Engine output: ", engineOutput

# Run the Stockfish engine
runStockfish()

---

import osproc, strutils, os, weave, selectors

# Task to read from Stockfish
proc readFromStockfish(stockfish: Process) =
  var selector = newSelector()  # Create a selector to manage non-blocking I/O
  selector.register(stockfish.outputStream)

  while stockfish.running:
    if selector.waitFor(1000):  # Wait for data to be ready, timeout in milliseconds
      let output = stockfish.outputStream.readLine()
      echo "Stockfish Output: ", output
      if output.contains("uciok"):
        echo "Stockfish is ready in UCI mode"
      elif output.contains("bestmove"):
        echo "Best move: ", output

# Task to write commands to Stockfish
proc writeToStockfish(stockfish: Process) =
  # Send UCI initialization command and other commands in a loop
  let commands = @["uci", "position startpos", "go depth 10"]
  for command in commands:
    stockfish.inputStream.writeLine(command)
    stockfish.inputStream.flush()  # Ensure the command is sent
    echo "Sent Command: ", command
    sleep(1000)  # Wait for a second between commands

# Main function to launch Stockfish and interact with it in parallel
proc main() =
  # Start the Stockfish engine
  let stockfish = startProcess("/path/to/stockfish", options={poUsePath},
                               stdin=stdinPipe, stdout=stdoutPipe)

  # Spawn tasks to interact with Stockfish
  spawn readFromStockfish(stockfish)
  spawn writeToStockfish(stockfish)

  # Run the weave event loop to process tasks
  runLoop()

  # Close the Stockfish process after tasks are done
  stockfish.close()

# Run the main function
main()

[tissatussa]

and what about the Nim package Weave, can it be of any use for me ?

Weave: A flexible parallelism framework in Nim that allows you to schedule and manage tasks and data-parallel work efficiently : https://github.com/mratsim/weave

btw. ChatGPT mentioned a few other Nim modules like this, but i couldn't get them working ..

[c-blake]

I don't know anything about Owlkettle. This all sounded familiar to me and I guess I already gave you some advice in this Nim Forum thread about a year ago which I still think is the simplest way to go for a beginning programmer. Though, as mentioned in the Forum, optimization to run stockfish as a coprocess rather than re-starting stockfish all the time is possible, it is also tricky/advanced and it sounds like you are struggling with basics.

To run the engine in parallel with the GUI, you could just send the output of stockfish to a file and then check every 10ms or so for the file growing to greater than zero (or whatever initial) size. Then you don't need threads. On Unix it could look as easy as popen("stockfish>/tmp/o &", "w") and then some writes to init & load the board state. All you need is a way for your GUI to do something every some-number-of-milliseconds. As I said, I don't know Owlkettle (nor do I want to learn right now), but this is pretty common in GUIs. Relevant keywords here are "alarms", "timers", and maybe "timeout" (the last often means to cancel an operation like pondering rather than "continue to wait", but it can be the right API to use since the timeout might apply to "waiting for user-keyboard/mouse input" not the "waiting for stockfish" side).

Relevant for the doing "something" of my above paragraph would be std/os.getFileInfo("/tmp/o").size to check if stockfish is done pondering. And then you need to read & parse its output into some Nim types that are part of your GUI. I don't know UCI (and sorry, I am not interested in learning right now), but I cannot imagine they have no command to "initialize the whole board to foo" or a command to "print out the new board". That is all you need. (Well, and the Nim types to represent, Nim code to parse & Nim code to format for stockfish/UCI, and all that other Nim code for the GUI.)

Your program must maintain that board state anyway (e.g., to not let users drag pieces from an empty square to another empty square). So, in addition to the types I mentioned above (or nearby variants), you will probably want (at least!) some kind of type Piece = enum King="K", Queen="Q"... and also type BoardState = array[8, array[8, Piece]] to update and render {or I don't know, maybe the two array index slots would be enums like array[A..H, array[1..8, Foo]] for some other enum with A..H as enum name tags }.

I really can only give a sketch and vague pointers on how to keep your project simple or within scope for you. In fact, I've almost run out of such pointers. I cannot write your program for you, though I encourage you to persevere and get to some nice usability point. :-) { And ChatGPT almost always disappoints me. }

As for the Nim destructor bugs -- yeah.. They happen. It sounds like the Owlkettle guy worked around it, but you may run into others.