Replace cutechess-cli by c-chess-cli in the worker?

[vdbergh]

cutechess-cli depends on the Qt framework and hence is a very heavy dependency of the worker. c-chess-cli is much more lightweight.

[FredFerguson01]

Does it work well on Windows ? This is from its description - "Windows support was added recently, and is still experimental at this stage."

[ppigazzini]

Windows support seems to be still a work in progress (msys2 mingw64 shell):

$ python3 make.py -c x86_64-w64-mingw32-gcc -t test
% rm -f c-chess-cli c-chess-cli.1.log log out1.pgn out2.pgn stdout test/engine training.csv
% x86_64-w64-mingw32-gcc -I./src -std=gnu11 -mpopcnt -DNDEBUG -Os -ffast-math -flto -s -DVERSION=\"2021-07-11\" -Wfatal-errors -Wall -Wextra -Wstrict-prototypes -Wsign-conversion -Wshadow -Wmissing-prototypes src/bitboard.c src/gen.c src/position.c src/str.c src/util.c src/vec.c test/engine.c -o ./test/engine -lpthread -lm
% x86_64-w64-mingw32-gcc -I./src -std=gnu11 -mpopcnt -DNDEBUG -Os -ffast-math -flto -s -DVERSION=\"2021-07-11\" -Wfatal-errors -Wall -Wextra -Wstrict-prototypes -Wsign-conversion -Wshadow -Wmissing-prototypes src/bitboard.c src/gen.c src/position.c src/str.c src/util.c src/vec.c src/engine.c src/game.c src/jobs.c src/main.c src/openings.c src/options.c src/seqwriter.c src/sprt.c src/workers.c -o ./c-chess-cli -lpthread -lm

Run tests:
% ./c-chess-cli -each cmd=./test/engine depth=6 option.Hash=4 -engine name=engine=1 option.Threads=2 -engine name=engine2 depth=5 -openings file=test/chess960.epd order=random srand=1 -resign count=4 score=13733 -draw number=40 count=3 score=11077 -games 965 -pgn out1.pgn 2 -concurrency 8 > /dev/null
The system cannot find the path specified.
% ./c-chess-cli -each "cmd=./test/engine 123" depth=3 -engine option.Hash=2 tc=10/0 -engine name=e2 tc=20/0 -engine name=e3 -sample freq=0.5 decay=0.05 resolve=y file=training.csv format=csv -openings file=test/chess960.epd -repeat -rounds 3 -games 30 -resign number=35 count=5 score=8192 -pgn out2.pgn 2 -log > stdout
'.' is not recognized as an internal or external command,
operable program or batch file.
% grep -v ^deadline c-chess-cli.1.log > log
grep: c-chess-cli.1.log: No such file or directory

File signatures:
% sha1sum stdout out1.pgn out2.pgn log training.csv
da39a3ee5e6b4b0d3255bfef95601890afd80709 *stdout
sha1sum: out1.pgn: No such file or directory
sha1sum: out2.pgn: No such file or directory
da39a3ee5e6b4b0d3255bfef95601890afd80709 *log
sha1sum: training.csv: No such file or directory

Overall signature:
% cat stdout out1.pgn out2.pgn log training.csv |sha1sum
cat: out1.pgn: No such file or directory
cat: out2.pgn: No such file or directory
cat: training.csv: No such file or directory
da39a3ee5e6b4b0d3255bfef95601890afd80709 *-

Here is the output of the test run:

$ ./c-chess-cli -each cmd=./test/engine depth=6 option.Hash=4 -engine name=engine=1 option.Threads=2 -engine name=engine2 depth=5 -openings file=test/chess960.epd order=random srand=1 -resign count=4 score=13733 -draw number=40 count=3 score=11077 -games 965 -pgn out1.pgn 2 -concurrency 8
[1] Illegal FEN 'p/8/8/8/8/PPPPPPPP/BRKNQBNR w HBhb - 0 1'
[1] Illegal FEN 'p/8/8/8/8/PPPPPPPP/BRKNQBNR w HBhb - 0 1'
[1] Illegal FEN 'p/8/8/8/8/PPPPPPPP/BRKNQBNR w HBhb - 0 1'
[1] Illegal FEN 'p/8/8/8/8/PPPPPPPP/BRKNQBNR w HBhb - 0 1'
[6] Illegal FEN 'pppp/8/8/8/8/PPPPPPPP/NRNQBBKR w KQkq - 0 1'
[1] Illegal FEN 'p/8/8/8/8/PPPPPPPP/BRKNQBNR w HBhb - 0 1'
[5] Illegal FEN 'R w HBhb - 0 1'
[3] Illegal FEN 'kq - 0 1'
[6] Illegal FEN 'pppp/8/8/8/8/PPPPPPPP/NRNQBBKR w KQkq - 0 1'
[8] Illegal FEN '/PPPPPPPP/RNBQKNRB w KQkq - 0 1'
[7] Illegal FEN '8/8/8/8/PPPPPPPP/RQKNBBRN w GAga - 0 1'
[2] Illegal FEN 'ppp/8/8/8/8/PPPPPPPP/NRNQBKRB w GBgb - 0 1'
[1] Illegal FEN 'p/8/8/8/8/PPPPPPPP/BRKNQBNR w HBhb - 0 1'
[5] Illegal FEN 'R w HBhb - 0 1'
[4] Illegal FEN 'PPPP/NRKRNQBB w DBdb - 0 1'
[3] Illegal FEN 'kq - 0 1'
[6] Illegal FEN 'pppp/8/8/8/8/PPPPPPPP/NRNQBBKR w KQkq - 0 1'
[8] Illegal FEN '/PPPPPPPP/RNBQKNRB w KQkq - 0 1'
[7] Illegal FEN '8/8/8/8/PPPPPPPP/RQKNBBRN w GAga - 0 1'
[2] Illegal FEN 'ppp/8/8/8/8/PPPPPPPP/NRNQBKRB w GBgb - 0 1'
[1] Illegal FEN 'p/8/8/8/8/PPPPPPPP/BRKNQBNR w HBhb - 0 1'
[5] Illegal FEN 'R w HBhb - 0 1'
...

I'm wondering if some SF Windows developers could help @lucasart sorting out the last problems :)

[vondele]

I'm using c-chess-cli quite regularly now, as it has some nice properties. From time to time there are still small bugs. Eventually, I could imagine using it, right now is probably a bit to early.

[NightlyKing]

I agree that eventually using c-chess-cli would be quite nice. It seems to be able to handle large concurrency and really low time controls quite a bit better. As pointed out in some recent discussions in discord, it lacks proper error handling/output which could be a pain in the future if that's not taken care of. If we were to have a loose-ish roadmap for fishtest we could include c-chess-cli and document what needs to be done to be ready for the switch.

[vdbergh]

It would be useful to give actual examples of the lack of "proper error handling". Better yet, post them as issues on the c-chess-cli github repository.

[vondele]

yes, agreed.

I'm aware of two features we would need, or take into account:

• ability to take a specific starting point in a randomized opening book (i.e. so we can have a randomized opening book, without duplicates on various workers)
• we still use pgn opening books. We probably would need to switch to .epd books, that's not too much of a big deal.

[ppigazzini]

On Windows msys2 python3 make.py -c x86_64-w64-mingw32-gcc -t test and python3 make.py -c clang -t test still have the problems reported in July.

[vdbergh]

Not going to happen. Closing.

[vondele]

experiments are ongoing to replace by https://github.com/Disservin/fast-chess That projects seems to have quite some momentum now, and is essentially feature complete for a drop-in replacements. Please can add links to their pre-existing fishtest branches.

[ppigazzini]

This issue is about c-chess-cli. Please, open a new issue for fast-chess