search

NethermindEth / Paprika

A custom storage engine of Nethermind, benefiting from the alignment of the underlying data structure with the layout of State & Storage trees of Ethereum.
GNU Lesser General Public License v3.0
91 stars 14 forks source link

Platform specific prefetcher #396

Open Scooletz opened 1 month ago

Scooletz commented 1 month ago

This PR migrates the prefetching from SSE based to an OS specific solution. The prefetch happens on a separate thread that accumulates pages to be prefetched in a channel and schedules them in bulks. Similar implementation could be provided for not needing pages anymore, but it would require either nop implementation on Windows or a better understanding of advising on Win. DiscardVirtualMemory cannot be used as it destroys the content which is not what is needed here. What is needed is a hint that the given area won't be used anymore.

The following methods are used to hint that given addresses will be used:

  1. PrefetchVirtualMemory on Windows
  2. madvise on Linux

While testing stats gathering, issuing prefetches when .Accept on a page is called increased the IO throughput significantly.

If only we had https://github.com/dotnet/runtime/issues/59776 in .NET itself 😭

Scooletz commented 1 month ago

@dipkakwani Summing up: maybe SSE prefetch shoudl be still available as a soft version and should be able to be used depending on the call site (DataPage)

github-actions[bot] commented 3 weeks ago

Code Coverage

Package Line Rate Branch Rate Health
Paprika 84% 79% âž–
Summary 84% (4743 / 5631) 79% (1577 / 1986) âž–

Minimum allowed line rate is 75%