search

graymauser / btcposbal2csv

List all bitcoin addresses with positive balance.
BSD 3-Clause "New" or "Revised" License
268 stars 201 forks source link

Error: unable to decode 831061 transactions #8

Closed komodin closed 3 years ago

komodin commented 5 years ago

Hi, I'm using bitcoind v0.18.0 and I have the node up2date.

When I run the btcposbal2csv.py, it process up to transaction #58027800 before die with the error "unable to decode 831061 transactions".

(btcbal) [bitcoin@btcd btcposbal2csv]$ python btcposbal2csv.py /home/bitcoin/.bitcoin/chainstate/ addrs2.txt
reading chainstate database
inmem
 parsed transactions: 58027800
unable to decode 831061 transactions
totaling 24371245714994 satoshi
writen to addrs2.txt

I have enough RAM (the process does not die with an out of memory error). At the moment of the unexpected exit, is just using around 6gb of memory.

Do anyone have any idea of what can I check? Thanks

graymauser commented 5 years ago

Hi @komodin what you are seeing isn't an error at all. It is just to state that there are 831061 unspent transaction outputs, in the UTXO set, for which there is no easy way to "decode the address". The address itself, as usually understood, is only proxy for humans...

For more information see https://en.bitcoin.it/wiki/Transaction#Types_of_Transaction.

komodin commented 5 years ago

Hi @graymauser, thanks for your response.

As far as I understand, the current number of total transactions is around 430 million (as you can see on blockchain.com), and the script is just parsing around 58 million TX.

Do I have any way to avoid the script exit when it found this behavior?

Thanks in advance!

graymauser commented 5 years ago

Hi @komodin this script processes only the UTXO transaction set -> These are the unspent transaction outputs - not all transaction (ie. https://www.blockchain.com/charts/utxo-count).

However, if you have up to date utxo set then there is some problem, because the total number of transactions should be about 80M now.

I'll investigate when I have more time, however, this can be related to segwit or something like that.

komodin commented 5 years ago

Thanks!

graymauser commented 5 years ago

Hi @komodin I've investigated a little and it looks that about 58 milion transactions is valid. When you run the bitcoin-cli with gettxoutsetinfo

you get something like this:

{
"height": 586378,
"bestblock": "0000000000000000001b8d1d8dcd720da68c61f53105b33ec7b04bbb0363e8c6",                                        "transactions": 33810842,
"txouts": 59089178,
"bogosize": 4447148006,
...
"disk_size": 3599864481,
"total_amount": 17829554.82325440
}

You want to look at the txouts entry which lists number of transaction outputs (i.e. spendable bitcoins) in the UTXO set.

And you can see that is is about the same number as what you are seeing in the tool output.

Now why the blockchain shows about 80 mil transactions ... I don't know.

There is a library which does lot of stuff related to bitcoin and also can return the addresses with unspent output see https://github.com/citp/BlockSci/issues/264. It appears that is can handle segwit transactions out of the box, so it might be interesting to compare...

Regarding the issue, I conclude that the scripts sees all the transaction outputs defined in the UTXO set - which should be complete set of unspent transaction outputs in the blockchain.