Open ytrezq opened 3 years ago
Let me guess the steps to reproduce this issue.
Can you confirm whether this is the case?
@riversand963 more exactly, I synced a full archival node using OpenEthereum which is a process taking 1 or 2 year to complete for building the database. OpenEthereum is using an old api and thus an old release of Rocksdb, and this is likely to never get updated.
Then using ldb
simply for reading updates the format and OpenEthereum can no longer use the database.
If WAL is enabled, then it is possible that the command (such as 'query') you used to read from the db tries to open the db, recovers data from the WAL, inserts them to memtable, and flushes. This can generate a new SST file, whose format may be too new for your old RocksDB release. Ldb commands, get or scan seem fine because they open the db as read-only and does not do flush.
Even then I d like to do things like adding sst files which require newer version of ldb while still using the resulting database with OpenEthereum.
Do you have a copy of the original DB to experiment with? If so, can you show the format_version
from the OPTIONS file in the DB directory?
$ grep format_version OPTIONS-*
format_version=4
Then using the newer ldb
you can try passing --try_load_options
, which should pick up the older format_version
from the OPTIONS file, and then should only write files compatible with the older RocksDB version.
Even then I d like to do things like adding sst files which require newer version of ldb while still using the resulting database with OpenEthereum.
FYI, I suspect we do not have good safeguards in place for calling new features like ldb ingest_extern_sst
while expecting them to be readable by an old DB. I wouldn't be surprised if those write manifest entries that are unreadable by older versions...
@ajkr : yes, I do have a copy of the original database and yes, I tried using --try_load_options
since an option file is present. The underlying Rocksdb version is 5.14.3
.
But after all, even the HISTORY file contains change which are said to not be optional and not forward compatible.
Expected behavior
The official Administration tools in this repository tries to keep database version.
Database administration shouldn’t means forced upgrade.
Actual behavior
even for just reading a single key without writing anything the footer version is updated which results in this error (because some software like OpenEthereum still use a RocksDb version which is several years behind and can’t be updated without a huge man power investment)
and prevents to use the old software.