Previous to this PR, a large amount of time was during validation was spent in the Difficulty Iterator, in particular deserializing and reserializing the PoW proof nonces. Since this information is not required to iterate through the difficulty of the past 60 blocks, a method of deserialization is introduced that deserialises a block header without its proof. Since the BlockHeader is unable to determine its correct hash without this info, the hash information is removed from the HeaderInfo struct. Further, the HeaderInfo Struct is renamed HeaderDifficultyInfo to reduce the potential for confusion (this struct should only be used to iterate through difficulty information)
For comparison, here is the current flamegraph of the included 100k header copy test (again on a 2.3 GHz 8-Core Intel Core i9) taking 36 Seconds (after the application of #3670) (53% of runtime spent in difficulty next iterator)
And with both this pr and #3670 applied - With the run taking 26 seconds (37% of time spent in same next fn)
Detailed Changes:
add deserialization_mode fn toReadable trait
Define DeserializationMode enum, with Full and SkipPow variants.
Add deserialization mode parameter to ser::deserialize function, and required modifications throughout the code and tests
Add deserialization mode to BinReader and required modifications throughout
Add methods to store and batch to return headers from the DB without their proof nonces
Modify proof deserialization to not deserialize proof nonces if the SkipPow mode is set on the reader
Remove block_hash from HeaderInfo struct, to ensure nobody attempts to rely on it being present later
Rename HeaderInfo to HeaderDifficultyInfo to better communicate its purpose
Small tweak to TUI stats where the block hash was being read from difficulty iterator, the stats will now need to fetch the block header again by height
Note since this is stacked we might need to do some rebasing wizardry to make this PR easier to view once #3670 is applied
Thank you for reviewing! Going to merge; there shouldn't be any consensus changes here and there's plenty of test effort for the next release incoming!
[note this PR is stacked on #3670]
Previous to this PR, a large amount of time was during validation was spent in the Difficulty Iterator, in particular deserializing and reserializing the PoW proof nonces. Since this information is not required to iterate through the difficulty of the past 60 blocks, a method of deserialization is introduced that deserialises a block header without its proof. Since the BlockHeader is unable to determine its correct hash without this info, the hash information is removed from the HeaderInfo struct. Further, the HeaderInfo Struct is renamed HeaderDifficultyInfo to reduce the potential for confusion (this struct should only be used to iterate through difficulty information)
For comparison, here is the current flamegraph of the included 100k header copy test (again on a 2.3 GHz 8-Core Intel Core i9) taking 36 Seconds (after the application of #3670) (53% of runtime spent in difficulty
next
iterator)And with both this pr and #3670 applied - With the run taking 26 seconds (37% of time spent in same
next
fn)Detailed Changes:
deserialization_mode
fn toReadable
traitDeserializationMode
enum, withFull
andSkipPow
variants.BinReader
and required modifications throughoutSkipPow
mode is set on the readerblock_hash
fromHeaderInfo
struct, to ensure nobody attempts to rely on it being present laterHeaderInfo
toHeaderDifficultyInfo
to better communicate its purposeNote since this is stacked we might need to do some rebasing wizardry to make this PR easier to view once #3670 is applied