Trying to import Caissabase-2022-12-24 database crash the software.

[chemtov]

Hi,

I downloaded the Caissabase-2022-12-24 database and converted it to pgn using Scid vs. PC version 4.23. So far so good. It resulted in a massive 3,5 GB file.

When I tried to import it, LucasR just crashed, at around 17000/19000 games imported.

I tried to import it in pychess, just to verify that the pgn was valid and it worked.

Then I splitted the pgn into smaller chunks of 18000 games, and then I was able to import ~180000 games. But, still some files made Lucas crashed. And the process through the GUI takes too much time to be able to import them all (7 242 070 games in there)

Nothing special in the debug.log file.

Regards,

[lukasmonk]

I have not been able to replicate the problem. I downloaded Caissabase, then installed Scid vs. PC, imported the file and exported to PGN, with the default options. I have count all Event tags in the pgn, and there are: 4.874.268

num = 0
with open("caissa.pgn", "rt") as f:
    for linea in f:
        if linea.startswith("[Event "):
            num += 1
print(num)

I have imported all the pgns in one go and with no crashes in 28 minutes with my old PC(an awful lot, but not a frequent operation):

[chemtov]

Hi,

hum ... maybe the system is the difference. I'm running the Linux version (on Fedora 37) from the website. Started with the ./LucasChessR2_05d_LINUX_WAYLAND.sh version and updated it to R2.06b through the interface. Both had the issue.

I'm trying to collect more information:

1. start lucaschess like this:

   cd ~/LucasChessR/bin strace -s 4096 -f -o /tmp/strace.log ./LucasR

2. create a new database and import the pgn

3. it end like this.

   Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway. Segmentation fault (core dumped)

Nothing in bug.log and the strace.log doesn't show anything special:

20103 read(35, "8. Rb .... ...[Round \"3\"]\n[Whi", 4096) = 4096
20103 lseek(35, 0, SEEK_CUR)            = 18258263
20103 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
20120 <... futex resumed>)              = ?
20119 <... futex resumed>)              = ?
20118 <... futex resumed>)              = ?
20117 <... futex resumed>)              = ?
20109 <... poll resumed> <unfinished ...>) = ?
20108 <... poll resumed> <unfinished ...>) = ?
20106 <... poll resumed> <unfinished ...>) = ?
20105 <... poll resumed> <unfinished ...>) = ?
20120 +++ killed by SIGSEGV (core dumped) +++
20119 +++ killed by SIGSEGV (core dumped) +++
20118 +++ killed by SIGSEGV (core dumped) +++
20117 +++ killed by SIGSEGV (core dumped) +++
20109 +++ killed by SIGSEGV (core dumped) +++
20108 +++ killed by SIGSEGV (core dumped) +++
20106 +++ killed by SIGSEGV (core dumped) +++
20105 +++ killed by SIGSEGV (core dumped) +++
20103 +++ killed by SIGSEGV (core dumped) +++

So, I used gdb to see the core:

#0  0x00007f7a23be52b6 in board_fenM2 () from /home/chem/LucasChessR/bin/OS/linux/FasterCode.cpython-38-x86_64-linux-gnu.so
#1  0x00007f7a23bec1bc in pgn_read () from /home/chem/LucasChessR/bin/OS/linux/FasterCode.cpython-38-x86_64-linux-gnu.so
#2  0x00007f7a23be2bfc in __pyx_pf_10FasterCode_9PGNreader_12__next__ (__pyx_self=<optimized out>, __pyx_v_self=<optimized out>) at FasterCode.c:6157
#3  __pyx_pw_10FasterCode_9PGNreader_13__next__ (__pyx_self=<optimized out>, __pyx_v_self=0x7f7a18cc07c0) at FasterCode.c:4708
#4  0x00007f7a3c148de0 in _PyObject_MakeTpCall (callable=0x7f7a23d9dba0, args=<optimized out>, nargs=<optimized out>, keywords=<optimized out>) at /opt/conda/conda-bld/python-split_1648465063888/work/Objects/call.c:159

So the problem seems to come from board_fenM2 in FasterCode.cpython-38

The trace continue on filed with a lot of things coming from "/opt/conda/conda-bld" which is surprising because it's not on my system, but I'm no gdb/cpython specialist, so I might miss something here.

Hope that helps moving this forward. If not, maybe you have an idea of what I could check/test/report next ?

[lukasmonk]

I believe that I have found the error. FasterCode.cpython-38-x86_64-linux-gnu.so.zip Unzip this file, and replace the file with the same name in bin/OS/linux.

[chemtov]

Awesome! It's fixed, the import worked perfectly.

Thanks you for the fix.

Regards,

[lukasmonk]

Perfect, thanks. The problem occurred with games with many moves, one of the variables was wrongly dimensioned. The funny thing and dangerous is that windows doesn't complain.