AutoSave thread to leave more room for other threads

[GoogleCodeExporter]

I looked at a thread analysis of a wallet that contains many 
txns/inputs/outputs to see why it is syncing so slow (2 blocks per second on a 
Nexus One).

It became clear that even with an autosave delay time of 1 second, the CPU is 
almost completely busy saving the wallet.

I think this is mostly because the delay isn't counted from the end of previous 
write but rather from the beginning. So if the write happens to take 1 second, 
the next write is already scheduled to begin. This is assuming there is a 
steady stream of wallet dirty events like while syncing.

Original issue reported on code.google.com by andreas....@gmail.com on 8 Apr 2013 at 9:24

[GoogleCodeExporter]

What kind of wallet takes a second to serialize and write? That must be huge.

We can fix the timing. For the root cause, I think we have to throw this one in 
with the "need another wallet format upgrade". We could do a quick hack to only 
save the wallet when the transactions in it actually change (and stop storing 
confidence as depth+work done but as height+nothing), but if the wallet is 
actually large then that implies it changes pretty often so it might not help.

Switching to a log-structured wallet with MultiBit style "replay management" 
would resolve this issue - we'd only append new records to the end when a 
transaction appears and checkpoint records can be added for work done.

Original comment by hearn@google.com on 9 Apr 2013 at 1:04

[GoogleCodeExporter]

On current master, the behaviour can still be experienced. I'm attaching a 
screenshot of a method profiling of a medium-sized testnet wallet.

Original comment by andreas....@gmail.com on 17 Sep 2014 at 9:50

[GoogleCodeExporter]

Original comment by andreas....@gmail.com on 17 Sep 2014 at 9:51

Attachments:

• [Screenshot from 2014-09-17 11:38:06.png](https://storage.googleapis.com/google-code-attachments/bitcoinj/issue-383/comment-3/Screenshot from 2014-09-17 11:38:06.png)

[GoogleCodeExporter]

I wonder if now that AbstractBlockChain can recover from wallets getting out of 
sync with "its" block store to some extent, can we reduce autosaving to a much 
smaller frequency?

SPVBlockStore stores 5000 blocks. If autosaving was done every 20 seconds even 
if download speed raises as high as 200 blocks per second the diff between 
wallets and chains should not exceed 4000 blocks.

Original comment by andreas....@gmail.com on 17 Sep 2014 at 10:06

[GoogleCodeExporter]

You could experiment with it for sure, but the underlying problem of course is 
that we're rewriting the entire wallet each time. There are lots of steps we 
can take to improve things with the eventual goal being a new wallet file 
format that scales a lot better. But it's a fair bit of work.

Original comment by mh.in.en...@gmail.com on 17 Sep 2014 at 10:30

[GoogleCodeExporter]

Switching from 1s to 10s seconds autosave delay reduced cpu usage of the 
autosave thread a lot, sometimes up to the point where it doesn't appear in the 
list of threads at all.

Overall, it doesn't help much with my mainnet wallet (there are more 
bottlenecks), but it should help with the average size wallet such as my 
testnet wallet.

Original comment by andreas....@gmail.com on 17 Sep 2014 at 12:59