search

Fantom-foundation / Carmen

Carmen engine for fast world state storage.
Other
18 stars 11 forks source link

Recovery fails with corrupted JSON file error #1018

Closed HerbertJordan closed 1 month ago

HerbertJordan commented 1 month ago

When stress-testing the checkpoint/recovery system for archives, the following issue was encountered:

Using checkpoint on block 13143.
Restoration complete. Exporting LiveDB genesis.
Forcing unlock of directory /tmp/db-heal-test/state_db_tmp_90448505/archive ...
Resetting archive in /tmp/db-heal-test/state_db_tmp_90448505/archive to block 13143 ...
Archive successfully reset to block 13143
2024/09/10 21:53:54 [t=   0:00] - export started
unexpected end of JSON input
exit status 1

The issue was produced running the following command:

python3 scripts/db-heal-test.py --aida /home/herbert/coding/fantom/aida --aida-db /media/herbert/WorkData/aida-db/mainnet --cp-granularity 1 --window 1

The issue seems to be created by the export step.

HerbertJordan commented 1 month ago

It looks like the corrupted file is the Archive's top-level meta file containing the root node and hash. This file is indeed not protected by a checkpoint.