Solution to "EgbbProbe not Loaded" in Linux/Mac (when looking to compile with nnue-probe) + Scorpio.ini suggestion

[kennyfrc]

Hi, since I'm using Mac, I had to compile scorpio myself (I also replicated the issue in Linux despite using install.sh). I found and was able to resolve what I think is an unintended bug (as it's not in the documentation).

After successfully compiling scorpio, egbbdll, and nnue-probe, when I run ./scorpio go quit, I get a EgbbProbe not Loaded error despite using the correct paths:

feature done=0
Number of cores 8 of 8
treeht 83886080 X 320 = 25600.0 MB
processors [1]
ht 67108864 X 16 = 1024.0 MB
eht 262144 X 8 X 1 = 4.0 MB
pht 32768 X 24 X 1 = 0.8 MB
EgbbProbe not Loaded!
loading_time = 0s
# rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
# [st = 13336ms, mt = 29250ms , hply = 0 , moves_left 10]
Segmentation fault: 11

After adding this logger in probe.cpp:247...

...
    hmod = LoadLibraryA(path);
    if (!hmod) 
      { fprintf(stderr, "dlopen failed: %s\n", dlerror()); 
        exit(EXIT_FAILURE); };
    if(hmod != 0) {
...

... and running ./scorpio go quit again, I get this error:

feature done=0
Number of cores 8 of 8
treeht 83886080 X 320 = 25600.0 MB
processors [1]
ht 67108864 X 16 = 1024.0 MB
eht 262144 X 8 X 1 = 4.0 MB
pht 32768 X 24 X 1 = 0.8 MB
dlopen failed: dlopen(/XXX-PATH-TO/egbbso64.so, 1): Library not loaded: libnnueprobe.so
  Referenced from: /XXX-PATH-TO/egbbso64.so
  Reason: image not found

The above was strange because egbbdll compiled correctly and it was linked to the nnue-probe library.

I was able to resolve it by moving libnnueprobe.so in the same folder as the scorpio binary.

feature done=0
Number of cores 8 of 8
treeht 83886080 X 320 = 25600.0 MB
processors [1]
ht 67108864 X 16 = 1024.0 MB
eht 262144 X 8 X 1 = 4.0 MB
pht 32768 X 24 X 1 = 0.8 MB
EgbbProbe 4.3 by Daniel Shawul
egbb_cache 4084 X 8216 = 32.0 MB
70 egbbs loaded !
Loading NNUE : morty0.1g-600k-d6.bin
NNUE loaded !
loading_time = 0s

Suggested long-term solution: add a path to nnue-probe in scorpio.ini.

[dshawul]

Yes, it is kind of complicated the way it is now. egbbdll.so has three dependencies, libnnprobe.so, linnnueprobe.so and libnncpuprobe.so. Moreover, there is no libnnprobe.so on the mac because tensorflow does not work there AFAIK. Anyway I am glad you are able to resolve this.

[kennyfrc]

Apple has been working on a tensorflow fork since Big Sur 11.0 (a month ago): https://github.com/apple/tensorflow_macos. It's still in alpha stage and still has a long way to go (couldn't make Lc0 work). And that's just the start of the problem as CUDA is not compatible with Apple-compatible GPUs/eGPUs. The best way forward if a Mac user wants to contribute resnets is through Google Colab. For now, NNUE+MCTS is the closest Mac users have to attaining a "human-like" sparring partner, which luckily Scorpio does best :) Thanks for maintaining this project.

Anyway, closing this now - just wanted to make a public record in case a Linux / Mac user encounters this issue.