search

dermesser / leveldb-rs

A reimplementation of LevelDB in Rust (no bindings).
Other
521 stars 59 forks source link

Fails to create a new iterator #14

Closed RadonCoding closed 2 years ago

RadonCoding commented 2 years ago

This is the error Error: Status { code: IOError, err: "IOError: snappy: corrupt input (expected stream header but got unexpected chunk type byte 200)" }

dermesser commented 2 years ago

As the error says, by all leveldb-rs can see, the data is corrupt. Is this a possibility in your use case? Does it happen deterministically with new databases?

Unfortunately I have not yet implemented any resilience measures, mostly because leveldb (the original) is not known for handling corruption particularly well, either.

RadonCoding commented 2 years ago

The database is not corrupted i tried with another crate rs-leveldb and it worked just fine.

dermesser commented 2 years ago

Oh, you opened a database not created by leveldb-rs with it? I've seen similar crashes before, but haven't investigated; most use cases just create their own DBs, which work. There is some bug lurking which detects some(?) foreign DBs as corrupt.

dermesser commented 2 years ago

I tested this a bit back and forth, and my suspicion of the bug being centered on the compression part seems to be confirmed. And within that, snappy itself complains (in leveldb-rs), whereas opening a compressed leveldb-rs database with original LevelDB may also stumble on the decompressed block contents (but works fine with uncompressed databases).

dermesser commented 2 years ago

Thank you for bringing this up - after 5 years I've finally managed to solve this bug. And it was quite trivial: I misused the snap crate's API, and used the framed format instead of pure snappy compression. A cursory test confirms that interoperability now works.

dermesser commented 2 years ago

fixed in 3d9a58510993460fa2d15a2ec0b5ec0abc95ed60