feature request: repair leveldb databases

omniscale / imposm3

Imposm imports OpenStreetMap data into PostGIS

http://imposm.org/docs/imposm3/latest/

Apache License 2.0

719 stars 157 forks source link

feature request: repair leveldb databases #118

Closed bdon closed 3 years ago

bdon commented 8 years ago

I'm running into an issue where my LevelDB databases are being corrupted. There seems to be a RepairDatabase function in the levigo library so it would make sense to have a command like "imposm3 repaircache -cachedir=/tmp/cache" or so. If you think this is a good idea I can look into implementing it.

olt commented 8 years ago

Why not, if it works...

How did you manage to corrupt the DB?

bdon commented 8 years ago

I'm not sure - it ran without issues for several months but is now reporting "Corruption: no meta-nextfile entry in descriptor". size of cache directory is 86GB

bdon commented 8 years ago

turns out that it was unrecoverable even with repair so I'm not certain if this is effective - closing

bdon commented 8 years ago

Actually got this working - success! some more details on failure states: https://docs.basho.com/riak/1.4.7/ops/running/recovery/repairing-leveldb/

It turns out the coords_index database was corrupted - I don't have any insight on how this happened but it was on a large amazon EBS volume.