Avoiding the esplora API for prevouts?

[shesek]

The docs mention that:

There is a few areas where a pruned node lacks the functionality needed that FN requires, so it smartly checks for that and if the node is pruned it falls back on Esplora over Tor v3; essentially only for parsing every input in your transactions for the verifier, if one of the inputs does not belong to you we fall back on esplora to fetch the raw transaction associated with that txid.

I'm wondering if this could be avoided. From a quick glance at the code, it seems that fetching the prevouts of non-wallet inputs is only used for determining the fee. If that's the case, it could potentially be avoided by taking the fee instead from:

• For outgoing transactions, it is available in the reply for listtransactions / gettransaction, under the fee field. (Note that this is only available if all the inputs in the transaction belong to the wallet, so it won't work if fullynoded supported coinjoin-style transactions.)

• For unconfirmed transactions, it can be retrieved via getmempoolentry.

For confirmed incoming payments, there's currently no way to tell the fee using a pruned bitcoin core node. But I would say that this is never actually interesting - for incoming payments you only really care about the fee to estimate how long confirmation is going to take, so no point in making this available to the user post-confirmation. I took a similar choice in bwt.

If there's some other reason to fetch prevouts that isn't related to fees, maybe consider to at least make it optional behind a flag?

[Fonta1n3]

Hey Nadav, thanks.

In addition to determining the fee we parse the prevout to get the address associated with each input to confirm whether the address belongs to us (as well as get detailed info about that address) as well as to fetch the amounts for each input.

fwiw bc its fetched tor to tor i don't see it as a privacy loss... but of course would be ideal to only use bitcoind.

[shesek]

we parse the prevout to get the address associated with each input to confirm whether the address belongs to us (as well as get detailed info about that address) as well as to fetch the amounts for each input.

What is this information used for? Just to show additional details when viewing a transaction, or is it required for the wallet's functionality?

If it's the former, maybe only load it when the user clicks to show the transaction's details? And also maybe have an option to disable it?

fwiw bc its fetched tor to tor i don't see it as a privacy loss...

The transactions/addresses don't get linked to the user's IP address, but do get linked to each other which is not ideal.

[Fonta1n3]

What is this information used for? Just to show additional details when viewing a transaction, or is it required for the wallet's functionality?

It is not required but verifying whether an input belongs to you or some unknown entity (and whether that input is a dust attack) is in my view a useful feature. Any time you tap a transaction to see its detail or create a transaction this information is shown to the user. If the wallet does own that address then the user will see very detailed info about that address and we can determine whether it is change and its label.

If it's the former, maybe only load it when the user clicks to show the transaction's details? And also maybe have an option to disable it?

I could definitely add a button saying "unable to fetch input details, tap to fallback on Esplora over Tor".

The transactions/addresses don't get linked to the user's IP address, but do get linked to each other which is not ideal.

Could you elaborate? Not sure I follow. All the Blockstream server would see is a get request with a random txid to its onion endpoint over Tor, beyond that there is no linkage of addresses to transactions of any kind other then the user seeing detailed info about the input which is the goal.

[shesek]

It is not required but verifying whether an input belongs to you or some unknown entity ...

You can determine if a given txid:vout belongs to the wallet or not by issuing a gettransaction <txid> request and looking through details for an entry with .category == "receive" && .vout == <vout>.

You only need a block explorer to get information about non-wallet prevouts, which doesn't seem necessary for what you're describing, if I'm understanding correctly?

All the Blockstream server would see is a get request with a random txid to its onion endpoint over Tor, beyond that there is no linkage of addresses to transactions of any kind

The blockstream server will see requests coming in from the same tor IP asking about various different transaction ids. They can understand that all the txids requested by the same IP within some time range are likely related to the same wallet, either directly as wallet transactions or as their prevouts.

A smart adversary even could analyze the requests and their timing, notice that there are requests coming-in at close proximity asking about a set of txids that match the prevouts of a certain transaction on the blockchain, and figure out that this certain transaction is a wallet transaction.

Switching Tor circuits between requests through Tor's control port could help (see example here), but I would personally prefer to have the option of avoiding this altogether. Also, switching circuits after every request will slow things down considerably.

[Fonta1n3]

You can determine if a given txid:vout belongs to the wallet or not by issuing a gettransaction <txid> request and looking through details for an entry with .category == "receive" && .vout == <vout>.

Right, the first command we make is gettransaction, if that fails we fall back on Esplora to fetch that inputs address and amount.

You only need a block explorer to get information about non-wallet prevouts, which doesn't seem necessary for what you're describing, if I'm understanding correctly?

It is not necessary unless you want to display the address and amount of each input.

I 100% agree with you and I think the way forward is to make it a global setting which defaults to the most private option (no esplora commands).

[Fonta1n3]

This has been added to master branch and will be released soon. Please do reopen if after the update you have any issues.

[shesek]

This is fantastic, thank you so much!

For future reference, this appears to be the relevant commit: https://github.com/Fonta1n3/FullyNoded/commit/a1fd8f491e52bf59d15a7c7ea1e77371c50faf94

Have you considered using getmempoolentry to complete the missing fee information for mempool transactions?

[Fonta1n3]

This is fantastic, thank you so much!

My pleasure, thanks for bringing it to my attention.

Have you considered using getmempoolentry to complete the missing fee information for mempool transactions?

Will include it in the next release.