search

Blockstream / electrs

An efficient re-implementation of Electrum Server in Rust
MIT License
321 stars 131 forks source link

Tx cache #110

Open RCasatta opened 2 months ago

RCasatta commented 2 months ago

Introduce a tx cache for the tx cache lookup.

The same crate for the cache has been introduced by romanz electrs, it's main advantages are that is a contiguous memory region capped in size.

The hypothesis is that with a big enough cache it's possible to run a public instance in lightmode, without duplicating tx data which is already present in the bitcoin node.

RCasatta commented 1 month ago

When lightmode is disabled, is there still an advantage to using a cache in addition to having the txs available in rocksdb?

yes, avoiding to hit the disk

RCasatta commented 1 month ago

Thanks for the review, I wasn't expecting it while in draft, but it's useful to have early feedback...

shesek commented 1 month ago

When lightmode is disabled, is there still an advantage to using a cache in addition to having the txs available in rocksdb?

yes, avoiding to hit the disk

I guess my question was, if we have all transactions available in a reasonably-fast in-process database and don't require network roundtrips to fetch them (as in romanz/electrs), would a cache still improve things in the typical case for typical access patterns? How large does the cache has to be for it to be beneficial? I'm not saying that it wouldn't help, just not so sure how to analyze this.

Another difference from romanz/electrs is that it needs to fetch full transactions in order to provide scripthash history, so a single history request can result in processing potentially hundreds or thousands of funding/spending transactions. In our implementation everything needed is already available in the history indexes, so we don't lookup txs internally as much.

Another thing to consider is that in the Esplora setup, we also have caching implemented at the HTTP level by caching proxies. For transactions, once they reach 10 confirmations we set a max-age of 5 years. However this does not apply to the Electrum server or to internal transaction lookup like a tx cache would.