search

spesmilo / electrum-server

Electrum server
MIT License
301 stars 388 forks source link

More guidance for indexing in memory. #168

Open nathaniel-mahieu opened 8 years ago

nathaniel-mahieu commented 8 years ago

In the how to for Electrum Server indexing in memory is listed as a way to speed up block indexing.

See this section, specifically the line creating /tmpfs: https://github.com/spesmilo/electrum-server/blob/master/HOWTO.md#step-7-import-blockchain-into-the-database-or-download-it

I am looking for a bit more guidance here. What needs to be placed in /tmpfs? Can electrum copy the block there for indexing? Does the entire bitcoin data directory need to live there? Or is it one of the electrum directories - where my leveldb needs to live.

Thanks, I'll pr the extra info once I get it working.

nathaniel-mahieu commented 8 years ago

Based on iotop and sysdig the location of leveldb was the high IO file.

Changing the leveldb location from /var/electrum-server/ to a 50G tmpfs file seems to have sped up my validation by about 30%.

Whats the best procedure for loading up the tmpfs with my saved index after reboots. Currently I am running cp /var/electrum-server/. /mnt/tmpfs -a prior to starting electrum server.

Whats the best procedure for saving the index before shutting down? My current thought is a cron every thirty minutes with something like rsync -ua /mnt/tmpfs/* /var/electrum-server/ Though a test of this indicates that it will take a long time, and it will compete for IO. Not sure how well it will work.

Thanks!

EagleTM commented 7 years ago

tmpfs is only meant to be used for initial catch-up and not really for continuously running it in there...