NethermindEth / Paprika

A custom storage engine of Nethermind, benefiting from the alignment of the underlying data structure with the layout of State & Storage trees of Ethereum.
GNU Lesser General Public License v3.0
91 stars 14 forks source link

Different fanout #397

Closed Scooletz closed 1 month ago

Scooletz commented 1 month ago

This PR amends layout of storage one more time. It makes StorageFanOut to be split into levels Level0, Level1, Level2, Level3. This allows to align it better with the Keccak -> uint mapping and use the id that Keccak is mapped to directly, without path appending.

The levels:

The crux lies in how the bits of uint id of the contract are consumed. They are consumed from the highest to the lowest, meaning that two consecutive ids like 1U and 2U will have the save Level1 and Level2 and will reside in two different buckets of the same page Level3. This should ease with navigation through the tree but at the same time make the db smaller for smaller networks (less contracts -> smaller number of buckets used on Level0 that spreads across highest 10 bits of uint id).

Ethereum mainnet

Importing Ethereum mainnet with Paprika.Importer shows a great reduction of application time (simpler logic for addressing, better inserts). This should also reduce the reading time as to get to an contract storage slot one need to go through 3 levels of the fanout pages (level0 is at the root) and then start querying. It should be top 5 levels of pages that should be scanned.

Import stats

image

CLI stats

image

github-actions[bot] commented 1 month ago

Code Coverage

Package Line Rate Branch Rate Health
Paprika 85% 80%
Summary 85% (4599 / 5414) 80% (1471 / 1831)

Minimum allowed line rate is 75%