search

interlay / interbtc-clients

interBTC Clients | Vault, Oracle, Faucet
Apache License 2.0
52 stars 30 forks source link

Limit number of parallel bitcoin rpc connections #396

Open sander2 opened 2 years ago

sander2 commented 2 years ago

Related to https://github.com/interlay/interlay.github.io/pull/275 .

At first I didn't understand how spancoin could run into the bitcoin connection limit of 16 when he was running only 7 vaults, as we use a blocking bitcoin api. However, I did some testing, and it shows that we indeed make several requests in parallel:

Oct 19 09:54:25.889 DEBUG bitcoincore_rpc: JSON-RPC request: getblockhash [2377248]    
Oct 19 09:54:25.967 DEBUG bitcoincore_rpc: JSON-RPC request: listunspent []    
Oct 19 09:54:25.967 DEBUG bitcoincore_rpc: JSON-RPC request: getblockhash [2377247]    
Oct 19 09:54:26.002 DEBUG bitcoincore_rpc: JSON-RPC error for getblockhash: RpcError { code: -8, message: "Block height out of range", data: None }    
Oct 19 09:54:26.080 TRACE bitcoincore_rpc: JSON-RPC response for getblockhash: "0000000045a081ced69de01bb65381e0c0e432cb43020bb629e35191e3638df0"    
Oct 19 09:54:26.080 TRACE bitcoincore_rpc: JSON-RPC response for listunspent: []

The reason that this happens is that while each task blocks on the bitcoin rpc, tasks can be run on different threads, resulting in this behavior. Maybe we should look into limiting the number of parallel connections, although I am a little bit afraid of unforeseen consequences such as timeouts in subxt..

spazcoin commented 2 years ago

I think it's totally ok for multiple threads to be blocking waiting for RPC responses. In fact, I'd encourage increasing the timeout so that they're willing to wait a little longer. Increasing the number of bitcoin RPC workers seems like a more appropriate scaling strategy. So I'd suggest: