search

btcsuite / btcwallet

A secure bitcoin wallet daemon written in Go (golang)
ISC License
1.13k stars 582 forks source link

Is there a way to add and monitor arbitrary addresses? #565

Open jordansaints opened 5 years ago

jordansaints commented 5 years ago

I have a need to monitor the account balances of arbitrary addresses for which I do not have the private key. From a waist-deep exploration of the code and docs available (few, sadly), I cannot seem to find a way to add an arbitrary address to an "account". Is this possible?

More details: Essentially I would like to use btcwallet as a caching layer of a given address to it's balances (at a given block height or confirmation count). Lots of chain explorers must implement this functionality, as querying for all txs for a given address is unbearably slow. Is this possible with btcwallet? Or is there another option I should explore?

totaloutput commented 5 years ago

@jordansaints will you be able to use the indexers in this case? Transaction-by-address (txbyaddridx) Index Creates a mapping from every address to all transactions which either credit or debit the address Requires the transaction-by-hash index It's documented here: https://github.com/btcsuite/btcd/tree/master/blockchain/indexers

jordansaints commented 5 years ago

@totaloutput thanks for pointing me to that code! I currently am using both --txindex and --addrindex on my btcd instance, which has gotten me 90% of the way there. The only issue now is that querying for the balance of a given address using SearchRawTransactionsVerbose still takes a very long time, especially when there are 1000s of transactions (even when run in parallel goroutines).

After more research I'm starting to think a more general-purpose cache/index would serve me better than a wallet. Maybe even the scantxoutset or gettxoutsetinfo RPC calls could help speed up this process, but AFAIK neither of these have been implemented in btcd. Am definitely open to any suggestions you have here :) Thanks!

l0k18 commented 5 years ago

If you are looking for better performance for address/balance searching, the only way I can think of to speed things up is to create new indices. I designed something a while back that stored height/txnum pairs in a string with cumulative in/out values. This would let you find all the needed blocks and transaction locations, and to be complete, a second balance value for values that have not passed various thresholds like coin maturity or minimum 6 balances and the like.

I would think to be safe, you would want the index to shop short of the tip, and only update the cumulative record for each address when there is no unresolved orphans at that height. A separate index could store the data in more verbose form with a hash of the tx hash and block hash and the values of each individual transaction, which you can then extract the values to update the immutable cumulative records.

As for how to implement it, on one hand you could just write jsonrpc queries for the blocks and slice them up using wire then add them to the index, or you could fork btcd, adding your custom query, which would also be a lot faster since it isn't marshalling and unmarshalling json in between processing.