search

franciscoBSalgueiro / en-croissant

The Ultimate Chess Toolkit
https://encroissant.org
GNU General Public License v3.0
658 stars 84 forks source link

Large database PGN file causes import to crash when importing #204

Open SaggingRufus opened 9 months ago

SaggingRufus commented 9 months ago

Describe the bug

I understand this is an edge case, but figured I would report it anyway.

I had a 20gb PGN I was trying to turn into a database (because there is no add games feature, bump to #191). After about 10 minutes of processing, the import failed, and none of the database was usable.

Reproduction

  1. Download every PGN from Lichess Elite Database
  2. Merge them all into 1 PGN cat ./* > merged-file.pgn
  3. Create a new database from the file

Platform and versions

Windows 10

Stack trace

Unfortunately I don't have one, I could perform the operation again and see if the same issue occurs though.
franciscoBSalgueiro commented 9 months ago

Did the database file not get created? If it was, you can inspect its contents with something like DB Browser, and see what was the last game to be converted.

yuval59 commented 9 months ago

image I tried reproducing the issue, but it worked fine for me. Could you try again maybe, and share a log or something? I'd like to check it but I can't reproduce the issue.

image This is the final size for my PGN file btw, is yours the same?

SaggingRufus commented 9 months ago

I will try again tonight or tomorrow and say the logs.