search

erthink / libmdbx

One of the fastest embeddable key-value ACID database without WAL. libmdbx surpasses the legendary LMDB in terms of reliability, features and performance.
https://erthink.github.io/libmdbx/
Other
1.16k stars 111 forks source link

I have a db from a disk snapshot that wont open #240

Closed ryanobjc closed 3 years ago

ryanobjc commented 3 years ago

I'm using erigon and I made a snapshot of a running erigon instance using google cloud persistent disks. The snapshot works by bisecting the io operations between the 'pre snapshot' and 'post snapshot', it should be similar to a hard power failure, except no partial block write.

I get this: [INFO] [11-11|03:53:33.215] Opening Database label=chaindata path=/home/erigon/.local/share/erigon/chaindata mdbx_validate_meta:13649 meta[2] used-bytes (1604902576128) beyond filesize (1604902391808), skip it mdbx_validate_meta:13649 meta[2] used-bytes (1604902576128) beyond filesize (1604902391808), skip it mdbx_validate_meta:13649 meta[2] used-bytes (1604902576128) beyond filesize (1604902391808), skip it mdbx_setup_dxb:15309 updating db-format signature for meta[2], txnid 773043, error -30779 [EROR] [11-11|03:53:33.221] Erigon startup err="mdbx_env_open: MDBX_PROBLEM: Unexpected internal error, transaction should be aborted, label: chaindata, trace: [kv_mdbx.go:217 node.go:531 node.go:534 backend.go:144 node.go:107 node.go:98 main.go:31 app.go:526 app.go:286 main.go:18 proc.go:255 asm_amd64.s:1581]"

erthink commented 3 years ago

Such snapshot is invalid and unrecoverable since you made it manually (e.g. copying on a fly from a live/running instance) instead of using the mdbx_copy utility. There is no way to recovery it.

@AskAlexSharov, FYI.

ryanobjc commented 3 years ago

I didnt copy it (eg using cp), I used gcloud PD snapshot, which works at the block/low level.

erthink commented 3 years ago

I didnt copy it (eg using cp), I used gcloud PD snapshot, which works at the block/low level.

It doesn't matter, because in essence the same thing happens: During the copying of a next portion of a DB file, the already copied part will be changed, as a result, the DB copy will be inconsistent.