NethermindEth / Paprika

A custom storage engine of Nethermind, benefiting from the alignment of the underlying data structure with the layout of State & Storage trees of Ethereum.
GNU Lesser General Public License v3.0
91 stars 14 forks source link

DataPage smart write through #367

Closed Scooletz closed 1 month ago

Scooletz commented 4 months ago

This PR introduces a change in the way of handling overflown DataPages.

In current situation, when a page is full and there's no place to put the new key in it, it will be flushed down. The flushing down is a powerful mechanism but requires to iterate over the SlottedArray, move items down and delete them in the original page. If the DataPage is near the top, this situation may happen quite frequently.

This PR introduces an additional check using batch.WasWritten that allows to check if a child page was written during this batch. If a key does not fit to a page (SlottedArray.TrySet failed), is non-empty (can have a nibble truncated), a child page exist and WasWritten during this batch, it's a perfect opportunity to pass through this page and move down.

The expectation is that by limiting the amount of time spent in flushing down, we could greatly decrease the time of applying a block. This would help with a lot of cases where there's a bulk of updates for a given account but we don't want to flush a few layers of DataPages constantly. The disadvantage is that this limits LRU behavior a bit (values pushed down in the tree).

To be implemented and tested.

github-actions[bot] commented 4 months ago

Code Coverage

Package Line Rate Branch Rate Health
Paprika 85% 80%
Summary 85% (4208 / 4938) 80% (1357 / 1692)

Minimum allowed line rate is 75%

Scooletz commented 1 month ago

Introduced and handled in #375