search

Blockstream / electrs

An efficient re-implementation of Electrum Server in Rust
MIT License
321 stars 131 forks source link

Optimize txo lookup #76

Closed shesek closed 3 months ago

shesek commented 8 months ago

From https://github.com/facebook/rocksdb/wiki/MultiGet-Performance:

There is a lot of complexity in the underlying RocksDB implementation to lookup a key. The complexity results in a lot of computational overhead, mainly due to cache misses when probing bloom filters, virtual function call dispatches, key comparisons and IO. Users that need to lookup many keys in order to process an application level request end up calling Get() in a loop to read the required KVs.

By providing a MultiGet() API that accepts a batch of keys, it is possible for RocksDB to make the lookup more CPU efficient by reducing the number of virtual function calls and pipelining cache misses. Furthermore, latency can be reduced by doing IO in parallel.

jamesdorfman commented 8 months ago

These optimizations make sense to me. utACK 5913e22ae4be667458c35d3b23cdc3e381bfe5b6.

shesek commented 6 months ago

Rebased.

RCasatta commented 3 months ago

utACK 4f037c4d9885215ce73de53003756d2e64ec766e

RCasatta commented 3 months ago

Needs rebase