bitcoinjs / indexd

An external bitcoind index management service module
ISC License
53 stars 23 forks source link

Low disk space limit setting #33

Closed unsystemizer closed 6 years ago

unsystemizer commented 6 years ago

I ran out of disk space while building index, which partially corrupted my Bitcoin data.

It'd be nice if indexd had a "low disk space" setting so that we could have it auto-shutdown before it gets a chance to do damage.

dcousens commented 6 years ago

I suppose we should have a canonical method for that, though I've never run into the issue of it causing corruption??? Maybe this is a bitcoind bug, if the data was lost there?

leveldb is atomic enough that you shouldn't have corruption there.

unsystemizer commented 6 years ago

indexd kept adding data and disk was full before other apps and OS could flush their pending writes, so if it only stopped before X percent or GB were gone that would do it independently of what other apps do. Maybe other apps would still fill up the disk, but we can’t change what other apps do.

It seems diskspace is a popular module for this. https://gist.github.com/jonmaim/eff4f6308c55c48ed97c83e390504625

dcousens commented 6 years ago

For reference

var disk = require('diskusage')

// ...

disk.check('/', (err, info) => {
  if (err) return ...
  if (info.available / info.total < 0.99) return ...

  // ... fail, 1% disk space remaining
}
dcousens commented 6 years ago

See https://github.com/bitcoinjs/private-bitcoin/issues/1 instead