search

bwt-dev / bwt

A lightweight wallet indexer for Bitcoin, available as an Electrum RPC server and a modern HTTP REST API.
https://bwt.dev
MIT License
275 stars 25 forks source link

Batch RPC calls to bitcoind #5

Open shesek opened 4 years ago

shesek commented 4 years ago

Not yet implemented in rust-bitcoincore-rpc: https://github.com/rust-bitcoin/rust-bitcoincore-rpc/pull/27

dpc commented 3 years ago

From my experience batching Bitcoin request at json-rpc layer is totally not worth it. Just use a threadpool.

shesek commented 3 years ago

Thanks for the tip!

Did you try with high-latency connections, say over Tor?

dpc commented 3 years ago

Did you try with high-latency connections, say over Tor?

No. The use cases I've worked with were all reasonably low latencies.

Just running this in my head, I don't see how batching would be better there either.

The optimization problem here is basically saturating IO throughput both on the network on disk of the fullnode (whichever becomes bottleneck first). Naive implementation blocking on each rpc call under-utilizes resources, wasting the time of the whole round-trip on each call.

Batching gets rid of most round-trips, but still suffers from the round-trip. Just once in N times instead of every time. A thread-pool can keep both the link/fullnode IO saturated 100% time. The amount of work (like serialization&deserialization) is roughly the same - with threadpool there's just more HTTP & JSONRPC envolope parsing, but these are minuscule.

On a long latency link like Tor, one would need more threads in the threadpool to stature the IO ... assuming high throughput. But since both the throughput and latency are (I assume) going to be low, then in practice a threadpool of 2 is going to saturate the connection.