A custom storage engine of Nethermind, benefiting from the alignment of the underlying data structure with the layout of State & Storage trees of Ethereum.
This PR introduces a change in the way of handling overflown DataPages.
In current situation, when a page is full and there's no place to put the new key in it, it will be flushed down. The flushing down is a powerful mechanism but requires to iterate over the SlottedArray, move items down and delete them in the original page. If the DataPage is near the top, this situation may happen quite frequently.
This PR introduces an additional check using batch.WasWritten that allows to check if a child page was written during this batch. If a key does not fit to a page (SlottedArray.TrySet failed), is non-empty (can have a nibble truncated), a child page exist and WasWritten during this batch, it's a perfect opportunity to pass through this page and move down.
The expectation is that by limiting the amount of time spent in flushing down, we could greatly decrease the time of applying a block. This would help with a lot of cases where there's a bulk of updates for a given account but we don't want to flush a few layers of DataPages constantly. The disadvantage is that this limits LRU behavior a bit (values pushed down in the tree).
This PR introduces a change in the way of handling overflown
DataPage
s.In current situation, when a page is full and there's no place to put the new key in it, it will be flushed down. The flushing down is a powerful mechanism but requires to iterate over the
SlottedArray
, move items down and delete them in the original page. If theDataPage
is near the top, this situation may happen quite frequently.This PR introduces an additional check using
batch.WasWritten
that allows to check if a child page was written during this batch. If a key does not fit to a page (SlottedArray.TrySet
failed), is non-empty (can have a nibble truncated), a child page exist andWasWritten
during this batch, it's a perfect opportunity to pass through this page and move down.The expectation is that by limiting the amount of time spent in flushing down, we could greatly decrease the time of applying a block. This would help with a lot of cases where there's a bulk of updates for a given account but we don't want to flush a few layers of
DataPage
s constantly. The disadvantage is that this limits LRU behavior a bit (values pushed down in the tree).To be implemented and tested.