ethereum / go-ethereum

Go implementation of the Ethereum protocol
https://geth.ethereum.org
GNU Lesser General Public License v3.0
47.71k stars 20.19k forks source link

core/rawdb: add accessors with reusable key buffers #30770

Open holiman opened 5 days ago

holiman commented 5 days ago

This came from an alloc dump on a running sync (mingeth). Screenshot 2024-11-20 at 09-55-10 tinygeth alloc_objects

Quite a lot of bytes are allocated temporarily when we write to db (batches), in order to construct the keys. These allocs are very shortlived, since the write-operations are not allowed to hold on to the keys, but instead copies it to some interrnal buffers.

This PR changes some of the most heavy mass-write-locations to reuse a key buffer. The locations are for example writing accounts received over snapsync, storage received over snapsync and merging in-memory snapshot-changes to disk.

Not sure how large effect this has. I suspect not a whole lot, but I'll run a benchmark sync and see if it anything is discernable.

Not high prio

rjl493456442 commented 4 days ago

Can you measure the reduction in memory allocations achieved by applying this change? Please evaluate its impact during both snap sync and full sync.

This pull request introduces optimizations to reuse the buffer for keying. These changes are only meaningful if they result in a obvious reduction in memory allocations.