Archiving of transactions

[GoogleCodeExporter]

Currently, bitcoinj holds all wallet transactions in memory, and the list gets 
longer an longer. As (future) HD wallets are meant for an eternity, I propose 
there should be some way to archive transactions.

My proposal:

When getting transactions via Wallet.getTransactions(), bitcoinj marks a number 
of them with a boolean Transaction.isArchivable(). Hard criteria for this might 
be "all outputs spent" plus "covered by at least one checkpoint". (Unconfirmed 
pending transactions could also be handled by this, if we find suitable 
criteria.)

A client can then add additional soft criteria like "at least a year old", 
maybe configurable by the user. Clients can move transactions out of bitcoinj's 
responsibility into an archive (or just throw them away), removing a ton of 
information that we don't need any more. In the case of Bitcoin Wallet, I'd 
probably only keep hash, time, amount and bitcoin address.

If the user accidently removes the archive, he only needs to replay the 
blockchain to restore.

Original issue reported on code.google.com by andreas....@gmail.com on 3 Mar 2013 at 4:04

[GoogleCodeExporter]

Yeah there's a guy on IRC who is wanting to implement something like that. I 
had proposed a different design to him, in which spent transactions are fed to 
an evictor object that just stores them "somewhere" and then we can provide a 
default evictor that appends them to a log file or something. That way you 
could seek backwards through the log to load older and older transactions. 
Other apps could put them into a database or something else. The guys nick is 
"muhoo". I'll point him at this issue.

Original comment by hearn@google.com on 3 Mar 2013 at 4:31

[GoogleCodeExporter]

Yes,that'd have been me.  In my use case, once a Transaction is accepted into 
the chain, all its TXOs are spent, and it's buried deep enough (or has enough 
work done), keeping it around is basically a memory leak for me, so I'll need 
to remove it somehow.

(I'll also need to age out old ECKeys too, for similar reasons, but that seems 
orthogonal. Just saying.)

I can see how each user of the library may want different algorithms for 
deciding when to consider a Transaction "safe" to remove, and my criteria may 
not make sense for anyone else, so it seems sensible to make it modular, with 
some default behavior.

I'm fine with implementing an TransactionEvictor or similar, as Mike described. 
Not sure when I'll get to it, but it's in my path now so it will have to get 
done.

Original comment by k...@bamfic.com on 3 Mar 2013 at 6:33

[GoogleCodeExporter]

The main point I was trying to make is that there are hard criteria that should 
be enforced by bitcoinj and soft criteria that are up to the client developer 
(and the user).

It should not be possible for example to remove a tx with unspent outputs (even 
if I set an all-permissive "Evicter"), because that would corrupt the wallet.

Original comment by andreas....@gmail.com on 3 Mar 2013 at 7:18

[GoogleCodeExporter]

It looks like Andreas ended up implementing this here:
http://code.google.com/r/andreasschildbach-bitcoinj/source/detail?r=f0ebc46850fe
e0a2b09d501eed1b9f9822ab0747

I ended up using a DB-backed system and avoiding the Wallet entirely.

Original comment by k...@bamfic.com on 8 Apr 2013 at 9:32

[GoogleCodeExporter]

Andreas' code just throws the transactions away, it doesn't store them to an 
archive.

Original comment by hearn@google.com on 9 Apr 2013 at 9:24