search

novitski / bitcoinj

Automatically exported from code.google.com/p/bitcoinj
Apache License 2.0
0 stars 0 forks source link

Another Re-Org Bug.... #468

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Had this re-org...

2013-10-05/17:58:34.207/UTC [New I/O worker #2] INFO  AbstractBlockChain[586] 
Re-organize after split at height 261906
2013-10-05/17:58:34.207/UTC [New I/O worker #2] INFO  AbstractBlockChain[587] 
Old chain head: 00000000000000076f8f8f52988bd639275d22775a86e710fce461c66288f329
2013-10-05/17:58:34.207/UTC [New I/O worker #2] INFO  AbstractBlockChain[588] 
New chain head: 0000000000000016d50a5244c888d57621e603b1d6f91dd3c8503f420f76f364
2013-10-05/17:58:34.208/UTC [New I/O worker #2] INFO  AbstractBlockChain[589] 
Split at block: 000000000000001335aaefcbd5ef4a4bb9842e4c6a6cc6de9bb7d92b66d2e297
2013-10-05/17:59:20.626/UTC [New I/O worker #2] INFO  Wallet[2231] Old part of 
chain (top to bottom):
2013-10-05/17:59:20.626/UTC [New I/O worker #2] INFO  Wallet[2233]   
00000000000000076f8f8f52988bd639275d22775a86e710fce461c66288f329
2013-10-05/17:59:20.626/UTC [New I/O worker #2] INFO  Wallet[2236] New part of 
chain (top to bottom):
2013-10-05/17:59:20.626/UTC [New I/O worker #2] INFO  Wallet[2238]   
0000000000000016d50a5244c888d57621e603b1d6f91dd3c8503f420f76f364
2013-10-05/17:59:20.627/UTC [New I/O worker #2] INFO  Wallet[2238]   
000000000000000516fd02261020a9f03ca50f13380bd5f2b1a3b332ce208677
2013-10-05/17:59:20.627/UTC [New I/O worker #2] INFO  Wallet[2305] 
depthToSubtract = 1, workDoneToSubtract = 639183349343662324
2013-10-05/17:59:20.735/UTC [New I/O worker #2] INFO  Wallet[2319] Replaying 
block 000000000000000516fd02261020a9f03ca50f13380bd5f2b1a3b332ce208677
2013-10-05/17:59:20.764/UTC [New I/O worker #2] INFO  Wallet[2319] Replaying 
block 0000000000000016d50a5244c888d57621e603b1d6f91dd3c8503f420f76f364
2013-10-05/17:59:20.765/UTC [New I/O worker #2] INFO  Wallet[2321]   tx 
0993ae39f61302f689741d18520091d5418321dc93546200f840cebfb7bc7675
2013-10-05/17:59:20.768/UTC [New I/O worker #2] INFO  Wallet[764] Received tx  
for -0.296 BTC: 
0993ae39f61302f689741d18520091d5418321dc93546200f840cebfb7bc7675 in block 
0000000000000016d
50a5244c888d57621e603b1d6f91dd3c8503f420f76f364
2013-10-05/17:59:20.768/UTC [New I/O worker #2] INFO  Wallet[780]   <-pending
2013-10-05/17:59:20.768/UTC [New I/O worker #2] INFO  Wallet[1020]   marked 
ad3b1bd3cd3b01eba6e4571c22c0b8c93728e5c2c62a8f4cc4e4949ef509d635:1 as spent
2013-10-05/17:59:20.768/UTC [New I/O worker #2] INFO  Wallet[1020]   marked 
764287e792361bd60301d041ce69b4ca76fbbbf33479158686c6c64c5473e24b:0 as spent
2013-10-05/17:59:20.768/UTC [New I/O worker #2] INFO  Wallet[951]   tx 
0993ae39f61302f689741d18520091d5418321dc93546200f840cebfb7bc7675 ->unspent
2013-10-05/17:59:21.006/UTC [New I/O worker #2] INFO  WalletFiles[112] Saving 
wallet, last seen block is 
261907/000000000000000516fd02261020a9f03ca50f13380bd5f2b1a3b332ce208677
2013-10-05/17:59:21.799/UTC [New I/O worker #2] INFO  WalletFiles[126] Save 
completed in 792msec
2013-10-05/17:59:21.800/UTC [New I/O worker #2] INFO  Wallet[2321]   tx 
ad3b1bd3cd3b01eba6e4571c22c0b8c93728e5c2c62a8f4cc4e4949ef509d635
2013-10-05/17:59:21.803/UTC [New I/O worker #2] INFO  Wallet[764] Received tx  
for 0.148 BTC: ad3b1bd3cd3b01eba6e4571c22c0b8c93728e5c2c62a8f4cc4e4949ef509d635 
in block 0000000000000016d5
0a5244c888d57621e603b1d6f91dd3c8503f420f76f364
2013-10-05/17:59:21.803/UTC [New I/O worker #2] INFO  Wallet[780]   <-pending
2013-10-05/17:59:21.803/UTC [New I/O worker #2] INFO  Wallet[1020]   marked 
a28513c3b0fcc77e19e366469647ff53a93ef6be517773c3070a77798ba20087:0 as spent
2013-10-05/17:59:21.803/UTC [New I/O worker #2] INFO  Wallet[951]   tx 
ad3b1bd3cd3b01eba6e4571c22c0b8c93728e5c2c62a8f4cc4e4949ef509d635 ->unspent
2013-10-05/17:59:21.825/UTC [New I/O worker #2] INFO  WalletFiles[112] Saving 
wallet, last seen block is 
261907/000000000000000516fd02261020a9f03ca50f13380bd5f2b1a3b332ce208677
2013-10-05/17:59:22.490/UTC [New I/O worker #2] INFO  WalletFiles[126] Save 
completed in 665msec

However ad3b1bd3cd3b01eba6e4571c22c0b8c93728e5c2c62a8f4cc4e4949ef509d635:1 was 
not actually marked as spent! And we just tried to double spend it.

Looking through code now to see where this bug is. 

Luckily I keep hourly backups of the wallet so will see what state the TX was 
in at 17:00 and 18:00 and 19:00.

Original issue reported on code.google.com by murexcon...@googlemail.com on 9 Oct 2013 at 6:48

GoogleCodeExporter commented 9 years ago 
I suspect I know what this is. The problem is the wallet isn't tracking the 
order in which transactions appeared inside a block, so if a single block 
contains two transactions, one of which spends the other, a replay might try 
applying them out of order.

I need to think about the best way to fix it. The ordering could be implied by 
making the transactions HashSet a LinkedHashSet and then ensuring all 
transactions are [de]serialized in the wallet file in order.

Original comment by hearn@google.com on 9 Oct 2013 at 7:01

GoogleCodeExporter commented 9 years ago 
Great! I noticed that: 
log.info("  {} {} <-unspent ->spent", tx.getHashAsString(), context);
in maybeMovePool is not being logged which I would expect too see.

Original comment by murexcon...@googlemail.com on 9 Oct 2013 at 7:14

GoogleCodeExporter commented 9 years ago 
The log shows the problem quite clearly - tx 
0993ae39f61302f689741d18520091d5418321dc93546200f840cebfb7bc7675 is replayed 
first, and it marks 
ad3b1bd3cd3b01eba6e4571c22c0b8c93728e5c2c62a8f4cc4e4949ef509d635:1 as spent 
correctly. Then ad3b1bd3cd3b01eba6e4571c22c0b8c93728e5c2c62a8f4cc4e4949ef509d63 
 is replayed.

I'll have a go at fixing this either tomorrow or Friday.

Original comment by hearn@google.com on 9 Oct 2013 at 7:16

GoogleCodeExporter commented 9 years ago 
Arh yes I see what you mean ad3b was moved to spent as expected. by 0993ae, but 
then when ab3b was in block too it moved it back out to unspent.

Problem I now have is a corrupt wallet that I need to refresh and try to resync 
with chain whilst there are still transactions coming in.

Original comment by murexcon...@googlemail.com on 9 Oct 2013 at 7:17

GoogleCodeExporter commented 9 years ago 
Is it worth to wait with 0.10.2 in order to include a fix for this?

Original comment by andreas....@gmail.com on 9 Oct 2013 at 8:13

GoogleCodeExporter commented 9 years ago 
We might as well try and get as many fixes as possible in, I suppose.

murexconsultant - that should work OK, I think, does the replay not work for a 
wallet of your volume?

If it doesn't, you could always replay the wallet against a local bitcoind that 
was disconnected from the network. That way it'll catch up with the chain and 
then you can reconnect it to the live network. But it shouldn't be necessary.

Original comment by hearn@google.com on 10 Oct 2013 at 8:23

GoogleCodeExporter commented 9 years ago 
Replay is fine just slow. But now I keep intermediate copies. So I have one 
from the last time I replayed (about a week ago) plus a chain that matches it.
What I then do is look for a quite moment run the replay on office PC connected 
to local bitcoind, but do not reset tx's just let it down load rest of chain 
(i.e. last week). Then the instant if finishes I kill the server process.
Then copy new wallet across to server and restart server. Hoping a block did 
not appear at wrong moment!

Then when the server starts up it deletes all spent txs older that 4032 blocks 
(storing hashes in db table used as lookup by listener). This keep wallet below 
10meg.

Feels a bit of a mickey mouse process.

This is why would like rdbms backed wallet as could then have just hacked out 
(with sql) that bad tx and reset to unspent the 30 tx'outs that were impacted. 
Would have been tricky but could have done with server up and running. But just 
don't have time at moment to come up with plan of how to refactor it all to 
allow this!

Original comment by murexcon...@googlemail.com on 10 Oct 2013 at 8:59

GoogleCodeExporter commented 9 years ago 
You could also write a tool that lets you edit the wallet file directly, by 
tweaking the spent flags.

Original comment by hearn@google.com on 11 Oct 2013 at 9:00

GoogleCodeExporter commented 9 years ago 
Proposed fix here:

https://code.google.com/r/hearn-bitcoinj/source/detail?r=c97a7808c58d0d3a3a483ef
af6c2329623787232

Review appreciated.

Original comment by hearn@google.com on 13 Oct 2013 at 10:18

GoogleCodeExporter commented 9 years ago 
I built BW versions including the patch

http://code.google.com/p/bitcoin-wallet/downloads/detail?name=bitcoin-wallet-3.2
2-bitcoinj0.10.2-test.apk

http://code.google.com/p/bitcoin-wallet/downloads/detail?name=bitcoin-wallet-3.2
2-bitcoinj0.10.2.apk

So far, it looks good. Cannot make out regressions.

Original comment by andreas....@gmail.com on 14 Oct 2013 at 6:22

GoogleCodeExporter commented 9 years ago 
This issue was closed by revision dfa722ccc84e.

Original comment by hearn@google.com on 15 Oct 2013 at 4:35

GoogleCodeExporter commented 9 years ago 
This issue was closed by revision e6ca742057a5.

Original comment by andreas....@gmail.com on 20 Oct 2013 at 3:28