experimental NNUE

[tissatussa]

hi, i see you're working on a new version of SunFish .. probably due to the late 4 Kb tournament ? nevermind .. i also see you're using a NNUE now, very small, and i'm curious .. i tried to run fancy.py but the explanation in the README does not resemble the folder structure, as i find it :

$ py tools/fancy.py -cmd "./sunfish_nnue.py nnue/models/tanh.pickle"

first, i guess py is your python3, ok .. then, tools/fancy.py means terminal should be in the root, where i also find ./nnue, indeed .. but ./sunfish_nnue.py does not exist in root, i only find the file ./nnue/sunfish_nnue_color.py .. i tried changing the command accordingly, but i don't succeed ..

how can i execute such command with a file & folder structure ?

[thomasahle]

Sorry, it seems the readme and the code was not in sync. Are things better now?

[tissatussa]

i see you changed the folder structure and file name(s), but now i have this error :

$ tools/fancy.py -cmd "./sunfish_nnue.py nnue/models/tanh.pickle"
Traceback (most recent call last):
  File "/home/roelof/Compiled/sunfish/tools/fancy.py", line 284, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/roelof/Compiled/sunfish/tools/fancy.py", line 234, in main
    engine = await load_engine_from_cmd(args.cmd, debug=args.debug)
  File "/home/roelof/Compiled/sunfish/tools/fancy.py", line 40, in load_engine_from_cmd
    _, engine = await chess.engine.popen_uci(cmd.split())
  File "/home/roelof/Compiled/sunfish/venv/lib/python3.10/site-packages/chess/engine.py", line 2681, in popen_uci
    await protocol.initialize()
  File "/home/roelof/Compiled/sunfish/venv/lib/python3.10/site-packages/chess/engine.py", line 1429, in initialize
    return await self.communicate(UciInitializeCommand)
  File "/home/roelof/Compiled/sunfish/venv/lib/python3.10/site-packages/chess/engine.py", line 1094, in communicate
    return await command.result
chess.engine.EngineTerminatedError: engine process died unexpectedly (exit code: 1)

[thomasahle]

What do you get if you just run "./sunfish_nnue.py nnue/models/tanh.pickle"?

[tissatussa]

that's a good test .. now i get this error :

$ ./sunfish_nnue.py nnue/models/tanh.pickle
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/usr/lib/python3/dist-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/usr/lib/python3/dist-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./sunfish_nnue.py", line 6, in <module>
    import numpy as np
  File "/usr/lib/python3/dist-packages/numpy/__init__.py", line 150, in <module>
    from . import core
  File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/usr/bin/pypy3"
  * The NumPy version is: "1.21.5"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

btw. this terminal command, and also the previous one, was done using an env with python3 .. must i install some required modules ?

[tissatussa]

i think i solved it .. i changed the top line :

#!/usr/bin/env pypy3

into

#!/usr/bin/env python3

and then it runs :

$ ./sunfish_nnue.py nnue/models/tanh.pickle
uci
id name sunfish nnue
option name EVAL_ROUGHNESS type spin default 13 min 0 max 50
uciok

[tissatussa]

this way also ./tools/fancy.py -cmd "./sunfish_nnue.py nnue/models/tanh.pickle" works .. so i guess it's solved!

[thomasahle]

What did you change? Just adding "./" before "tools"? That shouldn't make a difference...

Ah, you changed it to use python3 instead of pypy3. I wish there was a way to automatically detect which version of python was available and use that.

[tissatussa]

I wish there was a way to automatically detect which version of python was available and use that.

i don't know, and i never had the need to do such. You can put some info on this in the README ? To make Sunfish faster / stronger the user can do a few code changes ? I don't mind that and i think most users will (then) be able to.

[thomasahle]

You mean use python3 as default and have the user switch to pypy manually?

[tissatussa]

You mean use python3 as default and have the user switch to pypy manually?

yes, and write a short note about it.

[thomasahle]

Why use python3 as the default over pypy3?

[tissatussa]

Why use python3 as the default over pypy3 ?

i guess you know more about this .. for me Python is the mother and pypy a child - other speed increasing methods also exist. But i don't mind, as long as it's clear in the README for everyone.

[BlueCannonBall]

Why use python3 as the default over pypy3?

Pypy is quite obscure compared to good old cpython. I wouldn't expect your average python-capable system to have pypy3.

[thomasahle]

I wish there was a way to just run a script with the best python version installed :(

[zserge]

Well, technically it's possible but requires some shell scripting:

#!/bin/sh
""":"
for cmd in pypy3 python3 ; do
   command -v > /dev/null $cmd && exec $cmd $0 "$@"
done
exit 1 # no python interpreter found
":"""

import sys
print(f"python {sys.version}")