Explore bitcoind REST interface

[casey]

Using REST is also a more suitable interface for this application than RPC. It requires no authentication and can transmit blocks in binary form. It also does not have to parse a json request. It does require rest=1 in config though. However, it still deserializes a block to CBlock internally before sending. Ideally https://github.com/bitcoin/bitcoin/pull/26415 would be merged and this would be by far the fastest way to fetch blocks. See https://github.com/andrewtoth/rust-bitcoincore-rest.

Originally posted by @andrewtoth in https://github.com/casey/ord/issues/1502#issuecomment-1416828485

[casey]

Just going through my mentions and saw this again. I love the idea of using REST where we can. I find JSON-RPC very awkward, and don't really like the way it uses HTTP, but doesn't use HTTP verbs, headers, query parameters, status codes, and instead recreates those things in JSON.

I would love to see an initial simple use of the REST API merged, default-off, so we can start exploring it. We use HTTP a lot and I expect to expose an HTTP REST API, so it makes sense to use it internally.

[andrewtoth]

I don't think we need to make it an option. If the rest endpoint is responding we use it, otherwise continue with RPC.

[andrewtoth]

Most simple http libraries are async, which doesn't really work with the current architecture. I found https://github.com/oxigraph/oxhttp which seems to be what we need. Not sure if you have a better library in mind?

If you'd like to see the benefits, using ApacheBench and requesting via REST vs RPC an 85kb transaction f8af3c34c574e3039c47de0a32bce6b2753e4511337d0c9e13d31d7f2c83e374: Make sure you run with -rest=1 -rpcuser=user -rpcpassword=password for these

Benchmark REST:

ab -n 10000 -c 1 "http://127.0.0.1:8332/rest/tx/f8af3c34c574e3039c47de0a32bce6b2753e4511337d0c9e13d31d7f2c83e374.bin"

Benchmark RPC:

echo '{"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["f8af3c34c574e3039c47de0a32bce6b2753e4511337d0c9e13d31d7f2c83e374", 0]}' > /tmp/data.json
ab -p /tmp/data.json -n 10000 -c 1 -A user:password "http://127.0.0.1:8332/"

REST - 1.149ms mean request time RPC - 2.098ms mean request time

RPC also makes the CPU go up past 50% and the fan go on when I run the benchmarks, REST doesn't. Not sure how often we're calling this though and if less than 1ms savings per call is worth it? It might help reduce the JSON-RPC errors, but we can also tell users to increase rpcthreads and rpcworkqueue.