decred / dcrwallet

A secure Decred wallet daemon written in Go (golang).
https://decred.org
ISC License
216 stars 155 forks source link

wallet: Traverse by height in calcNextBlake256Diff #2292

Closed matheusd closed 11 months ago

matheusd commented 11 months ago

This commit improves initial chain sync performance by switching the calcNextBlake256Diff function to traverse the chain by height instead of following the header hashes.

calcNextBlake256Diff is used to calculate the difficulty required for block headers before DCP0011 is activated (which at this point in time is the majority of the chain). This requires traversing the chain backwards, down to the block prior to each work difficulty window change.

Previously this was done by following the header chain backwards. This involved loading and decoding each header in sequence, in order to extract the previous block hash, until each relevant block is reached.

This is a significant component of the cpu and memory loads for the initial chain sync of the wallet.

This commit improves this process by switching the traversal of the blockchain from being per block to being directly by height. Instead of loading and decoding each header from the db, only the necessary headers are accessed and only the required timestamp field is decoded instead of the full header.

This is safe to do because, as the existing comment in the ValidateHeaderChainDifficulties function explains, at the point of the execution of calcNextBlake256Diff, the ancestor chain of the tested header MUST be the wallet's current main chain. This implies that accessing the block hashes by height will return the correct ancestor nodes.

The net result (obtained via some ad-hoc profiling) is a reduction of the overall sync time of about 50 seconds (spent entirely in processing) and a decrease of about 15GB of allocated RAM (which further reduces GC pressure).

jrick commented 11 months ago

needs a rebase (or else i'll have to clobber your commit signature)