Created a new database, added 3 PGNs, during repair & compact database got corrupted

[gdinit]

Hi,

First of all thank you for maintaining SCID and improving with the version 5.

This is not a bug report but a question, please let me know if there is a better way of reporting this issue & asking for help? I've download & unzipped v.5.0.2 x64 on Windows. Then created a new database, added 3 PGNs, saved to the new database. After an hour of engine analysis & updates to the annotation (variations), I've issued a maintenance. During repair & compact database got corrupted, application crashed. It left behind COMPACT.xyz files (xyz being the 3 extensions).

Now I cannot open the database.

Is there something I can do to try and recover the seemingly corrupt database?

Thank you

[benini]

I used chatGPT to write this. It is a bit verbose, but I hope it is clear: you should have the untouched original database or a valid original_nameCOMPACT database.

In the chess software SCID (Shane's Chess Information Database), database compaction refers to the process of optimizing and reducing the size of the chess database file. Only after the database compaction process is complete and the compacted file is verified, SCID replaces the original database file with the newly compacted one.

If something goes wrong during the finalization step of the database compaction process in SCID, like a power shortage or unexpected interruption, it's crucial to recover the database and maintain data integrity. Follow these steps for recovery:

1. Assess the situation: Determine the extent of the interruption and identify any potential damage or data loss. Understand which specific step of the finalization process was interrupted.

2. Preserve the original database file: If the interrupted process left the original database file untouched, immediately create a backup. This ensures you have a copy of the original file as a fallback option.

3. Verify the integrity of the original file: Before proceeding with recovery, confirm the integrity of the original database file.

If you have the untouched original database or a valid original_nameCOMPACT database:

4. Rename the original_nameCOMPACT database: Change the name of the compacted database back to the original database's name.

5. Replace the faulty database file: Overwrite the interrupted or faulty database file with the renamed compacted database file. This restores the database to its previous state.

6. Verify the recovered database: Open SCID and ensure the recovered database functions correctly. Confirm that all game records, metadata, and other data are intact.

[gdinit]

Hello @benini and thank you for your quick response.

I completely understand the do-everything-first, only-then-replace-the-file approach. However, in this case, the COMPACT files were existent but sized 0 KBs. So I assume something must have gone wrong before the compact step kicked in.

In summary: (1)the original database files are no longer working. (2)COMPACT (interim) database files are empty.

Is there anything else to try here, any tools/command line utilities etc. or should I write this database off (i.e.: no recovery possible)?

[benini]

If you can upload the original database files somewhere (like wetransfer) and post the link I would like to investigate it.

[gdinit]

Hi @benini thank you so much for the offer but in a moment of weakness, seeing the 0 KB size in the COMPACT files, I've deleted them.

Since then, in another test database, I have realized that 0 KB in Windows Explorer does not mean completely empty file and some of them actually were less than 1 KB size but greater than 0 KB. In summary I should not have deleted the COMPACT files which is a lesson learned.

I am happy to stop working on this at this time because it was a test database with only 3 PGN single-game entries - I am OK to lose this particular "test" database.

I am back to SCID after many years of not playing chess at all & thus not needing SCID so my real worry was losing the "real" database which would contain many of my (future) games.

For now, as an interim measure, I will just take manual backups of the my real db time to time and hope the issue never repeats. If it does, however, I will then seek your help again and this time will provide the actual DB files and also the COMPACT files.

If you like please go ahead and close the issue now (& I can open a new one if the issue ever repeats).

Thanks again and best wishes!

[gdinit]

This is a closed issue, but just for posterity, I would like to report what I learned recently: it appears, the reason "compact database" operation was failing to delete the old file was a file lock.

Even though there was no annotation or engine was running (based on SCID GUI) it appears somehow a stray Stockfish engine process was holding a file lock on the database file. The engine I am using is Stockfish v.15.1, and the exact binary file is "stockfish-windows-2022-x86-64-avx2.exe".

If all those binaries are terminated, then SCID never experiences any issues with the compact operation.