Open minioff opened 1 year ago
I have been trying to play with high values (above 100) for bd-clients
in fulcrum and rpcthreads
in bitcoind respectively, but it doesn't seem to be making a difference. Can see in bitcoind log with -debug=rpc
that the (network) throughput is the issue. Is getrawtransaction
call parallelized?
Fulcrum is not designed to run in this way, with 1 machine so far away from another. This is because of limitations in how bitcoind RPC provides data. As a result, it is impossible to synch a full mempool due to the latency. Actually it synchs but I reckon it would take hours.
<SynchMempool> (Debug) downloaded 292 txs (failed: 0, ignored: 0), elapsed so far: 43.480 secs
It took 43 seconds to only download 292 txs. This is about ~6.79 txs/second which is on the order of your observed 200msec of latency (actually a little bit better than 200 msec per tx, at 150 msec per tx).
Fulcrum must do 1 RPC round-trip requesting the txn for EACH tx in the mempool. If you do the math, this means that for a 100k txn mempool (or whatever ridiculous thing is going on now on BTC) if we assume ~150msec per tx, it would take over 4 hours to synch a full mempool.
This is because there is no way for Fulcrum to retrieve the entire raw mempool in 1 download. The calls that bitcoind RPC provides do not allow for this. It must do a round-trip to bitcoin each time, issuing a getrawtransaction
call, for each mempool entry.
Is getrawtransaction call parallelized?
Not in Fulcrum.. if it were perhaps things would be #-of-threads faster. This would perhaps be a measure to speed it up but it's still impractical. Default bitcoin has an rpc work queue of 16. So even if you did 16 simultaneous calls -- it would still take 15 minutes to synch 1 very full mempool on your setup.
And that's assuming NO OTHER client asks bitcoind for stuff. If it does, bitcoind returns an error message when the work queue is full saying, basically, "sorry work queue full". So you can't even use all 16 work queue slots in this way for just mempool. You'd need to leave 4-10 slots free so we are looking at like 30 mins + just to synch the mempool initially.
Basically -- Fulcrum was not designed to work in such a high latency environment and it's impractical. Yes, parallel calls to getrawtransaction may help (to a point) since you can pay the fixed latency cost of 150 msec in chunks of 16 txns, say -- but it's still impractical.
Fulcrum should really be run with a local bitcoind instance.
Thanks for confirming my assumptions and observations.
Parallel getrawtx would indeed solve this issue as the machines can easily support 100+ threads.
Yeah it probably would, esp with a 100x speedup. I guess I just don't have the resources right now to devote to working on this. It is solvable, for sure, with parallelism. It would be quite an effort and for a niche case, for basically a "free" open source app, kind of too much work for me to do. :)
In certain conditions I are observing mempool fails to sync from remote bitcoind. The setup is
bitcoind on machine A fulcrum on machine A (connected to local bitcoind on A) fulcrum on machine B (connected to remote bitcoind on A)
The conditions to observe failure are a larger mempool (eg 100mb and above) and somewhat slow connection to remote bitcoind.
Working configuration with mempool decreased to 50mb I can observe
With mempool at 100 mb only this line is shown in the log
No further output regarding mempool operations is seen in the log. Mempool always remains empty.
Fulcrum is running with
--polltime 30 --bd-timeout 30
.