Newt error

[tissatussa]

hi, i use your engines in CuteChess and some run fine, but i have questions. I like to use Newt, while it seems to have time management, but after some moves it fails with an error .. i also ran Newt in terminal, using its default options (and turning off the opening book) .. doing go infinite in the start position gives this error : [ the same happens when i use python3 instead of pypy3 ]

$ pypy3 ptc_xboard.py newt
uci
id name Newt
id author Martin C. Doege
option name depth type spin default 4 min 0 max 1024
option name qplies type spin default 6 min 0 max 1024
option name pstab type spin default 1 min 0 max 1024
option name maxnodes type spin default 1000000 min 0 max 1000000000
option name usebook type check default true
option name matetest type check default true
option name UCI_Chess960 type check default false
uciok
ucinewgame
setoption name usebook value false
# usebook: False
go infinite
info depth 1 score cp 5 time 405 nodes 107 pv g1f3
info depth 2 score cp 0 time 644 nodes 190 pv b1c3 g8f6
info depth 3 score cp 5 time 1304 nodes 782 pv b1c3 g8f6 g1f3
info depth 4 score cp 0 time 2652 nodes 2246 pv g1f3 g8f6 b1c3 b8c6
info depth 5 score cp 4 time 5350 nodes 16414 pv g1f3 g8f6 b1c3 b8c6 e2e4
info depth 6 score cp 0 time 10842 nodes 57226 pv g1f3 g8f6 b1c3 b8c6 e2e4 e7e5
info depth 7 score cp 4 time 86493 nodes 717438 pv g1f3 g8f6 b1c3 b8c6 e2e4 e7e5 d2d4
Traceback (most recent call last):
  File "ptc_xboard.py", line 381, in <module>
    t, r = p.getmove(d, silent = True)
  File "/home/roelof/Apps/ChessEngines/TuroChamp/PyTuroChamp/newt.py", line 262, in getmove
    t, newPV = searchmax(b.copy(), MAXPLIES, aa, ab)
  File "/home/roelof/Apps/ChessEngines/TuroChamp/PyTuroChamp/newt.py", line 108, in searchmax
    t, vv = searchmax(b, ply - 1, -beta, -alpha)
  File "/home/roelof/Apps/ChessEngines/TuroChamp/PyTuroChamp/newt.py", line 108, in searchmax
    t, vv = searchmax(b, ply - 1, -beta, -alpha)
  File "/home/roelof/Apps/ChessEngines/TuroChamp/PyTuroChamp/newt.py", line 108, in searchmax
    t, vv = searchmax(b, ply - 1, -beta, -alpha)
  [Previous line repeated 1 more time]
  File "/home/roelof/Apps/ChessEngines/TuroChamp/PyTuroChamp/newt.py", line 106, in searchmax
    b.push(x)
  File "/home/roelof/.local/lib/pypy3.8/site-packages/chess/__init__.py", line 2216, in push
    assert piece_type is not None, f"push() expects move to be pseudo-legal, but got {move} in {self.board_fen()}"
AssertionError: push() expects move to be pseudo-legal, but got d2d4 in rnbqkb1r/ppp1pppp/7n/8/3Pp3/2N2N2/PPP2PPP/R1BQKB1R

i want to experiment with your code, eg. changing the parameters, but i also enounter problems with Bare. i'm familiar with Python and i read your doc :

Newt on the other hand counts plies normally from the root position, so maxplies and qplies should be even numbers.

At first this was something i missed .. setting different parameter values in CuteChess settings .. Note: i guess maxplies is wrong here - you mean depth ? because the Newt UCI info shows that option name ..

do you still work on the project ? maybe some bugs can be fixed easily ? or should go infinite not be used ?

[ i use a modern avx2 notebook on Xubuntu 22.04 ]

[mdoege]

Yes, I still work on the project, although the Nim version of Turochamp has been my focus recently, so I barely remember some of the details of the Python engines.

These Python engines are mainly intended for shallow search depths. So I would not really recommend to use "go infinite". Sooner or later they will probably run out of memory and crash. Or at least slow to a crawl.

But maybe you have found a bug in python-chess here? Obviously there is no d2 pawn to move in that position. Python-chess does most of the heavy lifting for these engines. So maybe check which version of python-chess you are using and upgrade to the latest release if necessary?

Yes, I suppose some of the engine/UCI variable names could definitely be better. I've kept them for, uh, historical reasons. Pull requests are always welcome, but I myself don't have a lot of free time for programming at the moment unfortunately.

The general idea is that most people will use engine default values more or less, because higher search depths are not really feasible with simple Python code like this. I recommend Sunfish if you want an amazing Python engine with high search depths. My engines are more like kindergarten-level chess engines I suppose... ;-)

[tissatussa]

I compiled and use your Nim version also .. it seems to run fine in CuteChess (sometimes only just 1 move before a loss by mate it makes an illegal move .. nevermind). Nim is new to me, I read it's some "advanced Python" which can also be compiled into a fast (?) binary, am i right ? Why did you switch from Python to Nim ? I'm eager to look into Nim, can it be combined with TKinter also, to build GUI's ? That would be very nice ..

Can you tell something about maxplies (maximum brute-force search depth) and qplies (maximum selective search depth) ? I read https://en.chessbase.com/post/reconstructing-turing-s-paper-machine : 6 counting rules can be applied to any position, but nothing is mentioned about qplies, i only read [...]three-ply search, which Turing used in his paper-and-pencil game[...] which seems incredible to me, because in many positions 3 ply means thousands of variations .. how was this possible for Alan Turing ? Do i miss something ? Did you add qplies yourself ? Does 'selective search' mean some 'pruning' is done to reduce the variation tree ? Then how does such pruning filter work ?

Another question is about the time management, while i guess this is needed to successfully play in CuteChess etc. Your PyTuroChamp docs show Newt has it, but it seems NimTuroChamp doesn't. Only a low depth / maxplies will do in CuteChess practice ..

I can check my version of python-chess, but i think the error is not related .. it might indeed be some memory issue at many nodes. I use SunFish also, even AngelFish ( https://github.com/VCHui/angelfish ) but i have problems with those : they almost hang my CPU, so they're not usable.

Thanks for your feedback.

[mdoege]

1. illegal moves: Like Sunfish, nimTuroChamp is a King Capture engine. Which means it does (almost) no checking of moves for legality. When it makes an illegal move, it should be interpreted as resigning by your chess GUI. This is also how the web version and the (currently inactive) lichess bot version handle this situation.

2. It's not really a "switch" to Nim per se, more like an alternate implementation that does not always play the same moves as the Python version because it works differently internally. I think the Nim version could be seen as slightly closer to what Turing could have programmed in his day: In Turing's paper, the King has a pawn value, so I think his engine would have used King Capture mechanics too. Whereas the Python engines only use legal moves, which is slower due to legality checking.

3. The reasons for trying out Nim were twofold: Firstly you can indeed compile a binary which runs very well on my Raspberry Pi 1, which I used for the lichess bot. The Python engine is too slow for Blitz/Bullet on the Pi 1. And secondly, Nim can compile to JavaScript, unlike Python, so I could port my Nim engine to the web browser fairly easily. And yes, I think Nim could be described as something resembling a compiled Python with explicitly typed variables. I don't think you can use Tkinter with Nim though. So far I have only tried SDL2 with Nim for graphics output, see my other GitHub projects. SDL2 is not a "proper" GUI library with widgets though...

4. qplies are the quiescence search plies, i.e. selective search plies. The Turing paper is a little vague about some details regarding selective search and "considerable moves", so each Turochamp implementation does it a little differently. In his paper games, Turing did all calculations in his head. But of course not even he could actually look at every possible move like a computer does. So modern Turochamp implementations will only reproduce about 50% to 60% of Turing's example game moves exactly, see https://mdoege.github.io/PyTuroChamp/comp.html But Turing's games were only meant as a proof of concept...

5. Yes, most of my engines lack time management. They are meant to mimic early historical engines which would also not have had that feature.

6. By the way, if you would like a Turochamp engine with deeper search than my programs, try https://github.com/herohde/morlock (written in Go). You can also play against it on lichess.

[tissatussa]

Very nice, thanks for this extensive answer! I will take a look at "SDL2", it's new to me. Recently i found Morlock and i tried to use it in CuteChess, but it plays horribly, eg. 1.Nh3 and 2.Rg1 ?! Something seems wrong here .. btw. i found the LiChess Bot list at https://lichess.org/player/bots but Morlock isn't there ?

[mdoege]

The lichess links are in the Morlock README: https://lichess.org/@/turochamp-1ply/all etc. Seems to be online right now.

My own bot is/was here: https://lichess.org/@/TuroBot/all

Note that my Nim lichess bot used an opening book. Without an opening book, these simple engines blunder horribly in openings. And sadly lichess players will then take advantage of these engines by playing the same winning games repeatedly to gain points. So in my view, a regularly updated opening book is a must for a lichess bot.

P.S. Yes, those Morlock moves look strange. Morlock searches deeper than my engines, so it should be stronger.

[tissatussa]

[ sorry to use this closed issue, but my question is related ]

i see you code Python and Nim. Recently i discovered Nim and i think i like it .. i managed to get some examples working .. now i focus on creating a GUI with Nim, until now i used TKinter with Python .. you mentioned SDL2 :

So far I have only tried SDL2 with Nim for graphics output, see my other GitHub projects. SDL2 is not a "proper" GUI library with widgets though...

well, i found several other GUI Frameworks, also with Nim, but this approach is rather new to me and it takes time to understand .. while learning and using TKinter, the many forum entries and examples were a great help for me .. TKinter is widely used .. therefor i found "Dear ImGui", see https://nimgl.dev/docs/imgui.html

what do you think ? is imGui a good alternative, compared to SDL2 ? do you recommend other GUI Frameworks for me to learn ? btw. i also looked at Electron because i do Javascript, but never created anything yet ..

[mdoege]

Yes, I think Dear Imgui is pretty nice. Immediate-mode GUI libraries in general are very en vogue right now. In my PyGame and Nim/SDL apps, I also redraw my whole GUI on every frame, e.g. see https://github.com/mdoege/Piano, so it is essentially the same approach. But Dear Imgui already has a lot of widgets predefined, so it is easier to use than "raw" SDL/PyGame.

The interfaces created with Dear Imgui do look a little like a "programmer interface" though, so if you are used to Qt, GTK, Tkinter, etc. they may appear a little too basic in comparison. So I am not sure if end users will be too impressed with Dear Imgui apps...

[tissatussa]

thanks for this info .. i really appreciate it .. i often read many comments and thus follow their links but this whole GUI Framework thing is rather new to me .. just as nodejs .. i'm basically a (old-school) PHP & JS & Python programmer, mainly web based .. there's so much new software, it's hard to follow .. so i'm glad you point me in a good direction .. as long as Imgui is stable and it offers my wanted functionalities, its layout is not an issue for me .. but, can styling be done with Imgui ? You tried Imgui yourself ? Can you point me to example code ?

[mdoege]

I checked out https://github.com/ocornut/imgui in early 2021 and tried some of their C++ examples. I have not tried Nim bindings yet, but I imagine they are similar.

There is support for themes in Dear Imgui, but more in the sense of changing interface colors. So don't expect color gradients or fancy styling like CSS styling on the web. Dear Imgui is fast but not very pretty...