LevelDB corruption not handled

SlexAxton commented 11 years ago

No idea how things got corrupted, but my best bet I think is to just reinstall, figured I'd paste this here in case there can be a --resync command or something that just wipes out corrupted stuff or something. No worries if not, I'm sure a reinstall will help.

$ npmd --sync

events.js:72
        throw er; // Unhandled 'error' event
              ^
OpenError: Corruption: missing start of fragmented record(2)
    at /Users/alexsexton/.nave/installed/0.10.13/lib/node_modules/npmd/node_modules/level/node_modules/levelup/lib/levelup.js:112:19

SlexAxton commented 11 years ago

(I spose killing ~/.npmd might have also done the trick, but it's too late to try now)

dominictarr commented 11 years ago

yeah, rm -rf ~/.npmd and then just run with --sync again will work.

what error where you getting? sometimes you see something about a LOCK file, which prevents two processes opening the same db at once, if you get a bad crash, and it's not deleted, you can just delete that.

iamkevinv commented 11 years ago

Having the same thing a lot over the weekend. Basically i'm blaming a flakey network.

dominictarr commented 11 years ago

oh, okay, this is really weird. Basically, this should just never happen - network or not. this is really a leveldown issue.

can you guys both post results to:

npm ls leveldown

dominictarr commented 11 years ago

oh, this might be the answer https://npmjs.org/package/leveldown#leveldown_repair

rescrv commented 11 years ago

Cross-posted from https://github.com/rvagg/node-leveldown/issues/52:

I'm very interested in fixing issue 197 on Google's bug tracker. If anyone has a DB exhibiting this corruption, contact me or follow up with information about where I may download the corrupted copy.

dominictarr commented 11 years ago

Okay, If anyone gets this error message again Please save your ~/.npmd directory into a tarball and upload it somewhere!

DamonOehlman commented 10 years ago

OK, I've managed to get a corrupted ~/.npmd folder, have renamed the folder and will push up an archive somewhere.

I don't know if lev can be held responsible, but I was trying to inspect the contents of the npmd leveldb data at the time using it. I'm expecting this occured during the key cache generation step of lev (based on hij1nx/lev#26) as I never made it to the repl.

DamonOehlman commented 10 years ago

Right, here's the archive of the corrupted ~/.npmd folder:

https://dl.dropboxusercontent.com/u/6319731/npmd-corrupt.tar.bz2

dominictarr commented 10 years ago

what error did you get?

DamonOehlman commented 10 years ago

OpenError: Corruption: 17 missing files; e.g.: /home/doehlman/.npmd/005714.sst
    at /home/doehlman/code/npmd/npmd/node_modules/level/node_modules/levelup/lib/levelup.js:112:19

dominictarr commented 10 years ago

Oh, that sounds serious. can you post this as an issue on levelup?

rescrv commented 10 years ago

I believe it's resolved by: https://groups.google.com/d/msg/leveldb/GXhx8YvFiig/FhMHoWz7MNAJ

rvagg commented 10 years ago

leveldown@0.10.2 is out with the unofficial fix for this osx corruption bug, I believe this issue can be closed now

dominictarr commented 10 years ago

cool, thanks!

dominictarr / npmd

LevelDB corruption not handled #10