search

nanocurrency / nano-node

Nano is digital currency. Its ticker is: XNO and its currency symbol is: Ӿ
https://nano.org
BSD 3-Clause "New" or "Revised" License
3.48k stars 783 forks source link

Split Block table #4053

Open clemahieu opened 1 year ago

clemahieu commented 1 year ago

Currently the block table maps block_hash -> block data. Each block is keyed on random data which causes frequent database page splits, hurting performance and causing internal fragmentation. For highest performance and lowest internal fragmentation, the database should append data whenever possible.

Splitting the block table in to one table mapping block_hash -> index and another table mapping index -> block_data significantly reduces the amount of fragmented data. Initial tests show disk space saving up to 50%, a speed up for non-seeking disks, and a significant speed up for seeking disks (HDDs).

Organizing blocks this way has the side effect of providing a topology sort of blocks which can be used for incremental ledger snapshots.

My1 commented 1 year ago

Initial tests show disk space saving up to 50%, a speed up for non-seeking disks, and a significant speed up for seeking disks (HDDs).

this is awesome to be honest especially for the smaller nodes.

Organizing blocks this way has the side effect of providing a topology sort of blocks which can be used for incremental ledger snapshots.

this is also very cool.

qwahzi commented 1 year ago

Related draft/work-in-progress PR for review & feedback: https://github.com/nanocurrency/nano-node/pull/4295