search

janoside / btc-rpc-explorer

Database-free, self-hosted Bitcoin explorer, via RPC to Bitcoin Core.
https://bitcoinexplorer.org
MIT License
1.58k stars 1.18k forks source link

Show the spending transaction of spent outputs #356

Open Pantamis opened 3 years ago

Pantamis commented 3 years ago

Solve #270

Requirements: Electrum server supporting Electrum 1.5 method blockchain.outpoint.subscribe

To activate this option you must set BTCEXP_ELECTRUM_TXINDEX = true in .env file as if you want to query ElectRS txindexing.

ElectRS 0.9.0 will support this method thanks to romanz/electrs/pull/446 This means this PR should not be merged until spesmilo/electrumx/pull/90 is merged

Screenshots time !

A transaction with many outputs to "bench" loading speed : 7cdaf30c42eac0271108fe0aa170a02a31f729fb10079156f06611f0489f8f67

BTC-RPC-Explorer needs to make one blockchain.outpoint.subscribe call for each spent transaction output (no call for UTXOs). There are around 250 spent outputs in this transaction at the time I wrote this PR, I load it in less than 40 secondes on my RPi (blockchain stored on HDD).

btc-rpc-explorer-mega-tx

A random transaction with outputs in the mempool at the time I wrote this PR (for easy copy-pasting txid = 6f949c749d890a250a40c29ca8c97576d47eca7f113980f65a0c62fb330516a1 )

btc-rpc-explorer-mempool-tx

The txid of spending transaction is clickable link and redirect to the explorer page of the sending transaction.

Electrum 1.5 specifications say that blockchain.outpoint.subscribe must subscribe the client to notification of changing state of the outpoint (spend, unspend, maybe reorg). We only need the output of this method so I unsubscribe the explorer immediatly after collecting it to avoid spamming the electrum server of subscriptions. ElectRS doesn't implement subscriptions for now so the unsubscribe method will fail, but this can be ignored.

Pantamis commented 3 years ago

Rebased !

Pantamis commented 3 years ago

0499abe add batch JSON request to electrum server to collect all spending transactions of all outpoints in one Electrum call.

I didn't use requestBatch defined in electrum-client lib

https://github.com/janoside/electrum-client/blob/d98e929028d2b05c88a41fe37652737912bead79/lib/client.js#L123-L144

My problem is that the second parameter must be the same for all batched JSON and I can't pass an outpoint as first argument into it because param is in a supplementary array. So I ended up rewriting the batched request in the electrum api, could be worth it to rather make this change in janoside\electrum-client then use requestBatch for batch json rpc here.

Pantamis commented 3 years ago

I notice that there exist transactions that have so many outputs that ElectRS takes too much time to answer so it may be worth adding some limitation on the number of outputs before showing the spending transaction of all outpoints.

An example of such heavy transactions: d22c53dd5a79fb6cc7a10ed6531761ae43ce1ce2ecd1ebbc5746ba82c71e6d70

Kixunil commented 3 years ago

No wonder, that transaction got blockstream.info (which uses even more efficient implementation) stuck too.

Could it be loaded asynchronously? Show the page, load outputs chunk-by-chunk (20 at a time or something) and show them as the data arrive. But ultimately I'd like it to be profiled and try to find if there's a reasonable optimization for electrs.

shesek commented 3 years ago

No wonder, that transaction got blockstream.info (which uses even more efficient implementation) stuck too.

Seems to work for me: https://blockstream.info/tx/d22c53dd5a79fb6cc7a10ed6531761ae43ce1ce2ecd1ebbc5746ba82c71e6d70

Kixunil commented 3 years ago

Ah, works for me too now. Was probably a fluke.

Pantamis commented 3 years ago

The last commit adds a limit of 200 to the number of outpoints of a transaction for which we fetch the spending transactions to show them along with the numbers of blocks between the transaction and the spending of an outpoint. This is to prevent the Electrum server from DDoS because of heavy transactions with many outputs (such transactions can generally be found as two hops spending transactions of coinbase outputs).

When there are too may outpoints, the explorer will just provide a link for each spent outpoint to a new page on the outpoint. This page will ask to the electrum server for the spending transaction and return its paget if it exists with a message telling the user which outpoint he is looking at and how many block between its creation and spending. If the spending transaction doesn't exist, it will return the funding transaction page with a message to inform the outpoint is not spent.

When looking at a transaction in the explorer from a panel not in a transaction page (from mempool section or blocks content or address history), we also provide a link to the outpoint page of each outpoint of each transactions shown. However, we cannot tell if an outpoint is spent in advance or not so the link presence doesn't garantee the outpoint is spent which is why it reads "Outpoint Status" (spent or unspent).

This reduces the need for https://github.com/spesmilo/electrumx/pull/90#discussion_r693485146 @Kixunil

Things are a bit mess up when the Electrum server doesn't support Electrum 1.5 (the explorer still display a link to outpoint statuseven if not supported...). I didn't find a good fix for now but it should be straightforward.