search

spesmilo / electrum

Electrum Bitcoin Wallet
https://electrum.org
MIT License
7.4k stars 3.08k forks source link

Reorganize storing of header files #4437

Closed haarts closed 6 years ago

haarts commented 6 years ago

This is not a bug report but refactor proposal.

I noticed that the headers of the active blockchain are stored in the file ~/.electrum/blockchain_headers(root might be different). This is somewhat confusing (for me). Electrum is an SPV wallet and connects to different servers. Each server is a full node with a particular version of the truth, a particular version of the blockchain. Electrum mirrors each truth by downloading headers (after the last checkpoint) from that server.

At the moment the code involved in managing the relation between an interface and a blockchain is not as easy as it could be (for example the on_get_header method).

Wouldn't it be easier to tie a header file on disk directly to a server? For example the file headers-tardis.bauerj.eu contains the headers we've got from the server 'tardis.bauerj.eu'. I think that this will make the code base much less complex.

An obvious downside is that we keep multiple header chains on disk. This is not a problem on a desktop machine but I don't know what the implications are for Android clients.

Also I'm unsure how the concept of a fork fits into this. I'm do not know what gives rise to the persistence of a fork on disk but I'll find out.

As I'm working on integrating libbitcoin I leave the current code intact and have my libbitcoin code try this idea. If it turns out to be equally opaque then nothing is lost (only my time). Perhaps this is a dumb idea, that's why I want to discuss it with the experts first.

SomberNight commented 6 years ago

There are dozens of servers, and the blockchain_headers file is ~40 MB on mainnet (though it's mostly zeroes before the last checkpoint, and thanks to filesystems supporting sparse files, it takes up a lot less hopefully). Still, on Android, you don't want to duplicate this file for each server. IIRC the blockchain_headers contains the headers of the most work chain -- not the headers from any server in particular, such as the currently connected one.

haarts commented 6 years ago

OK, so Android would be the show stopper here, otherwise it would have been a good idea?

I still find it hard to understand what 'most work chain' means. There are multiple 'most work chains'. But allow me to rephrase this in a way my brain understands it; blockchain_headers contains a continuous chain of headers sourced from, potentially, different backends. When receiving a header several things can happen:

  1. a received header references a previous header which we do not have.
    • Action: discard header.
  2. a received header leads to a fork b/c we already have a header pointing to the same previous header.
    • Action: create a fork.
  3. a received header neatly fits on the tip.
    • Action: extent blockchain_headers.

How does Electrum now deal with a switch, by the user, to a shorter chain? Or more generally; what does Electrum do when the user switches servers?

haarts commented 6 years ago

After a fruitful discussion on IRC I now better understand what's going on. My failure was to understand that when receiving a block and that block applies to a chain Electrum knows of it doesn't matter from where that block originates. Closing.