Get rid of non-relevant transaction inputs and outputs in wallets

[GoogleCodeExporter]

In order to fight the rising memory usage of bitcoinj-maintained wallets, I 
think the greatest lever is reducing the number of objects a tx is composed of. 
Most of the problematic transactions have either a huge amount of outputs or a 
huge amount of inputs (or both).

I suggest keeping only the relevant (to the wallet) in/outs. Why are we keeping 
them currently?

Non-relevant in/outs should not have impact on the value of the tx and 
therefore wallet balance should be unaffected. Importing addresses is currently 
not supported anyway and will probably never be. Relaying tx from the wallet is 
probably not useful (the memory pool is meant for this). Bitcoinj itself only 
creates relevant in/outs when creating a tx, so the resending feature should be 
unaffected.

What probably will happen is that tx are moved to the spent pool earlier. I 
suspect (but don't know) that currently tx are spent only if all outputs have 
been spent, including the non-relevant ones. Tx with a huge number of outputs 
will probably never be spent, defeating current (in my branch) wallet eviction.

Without further work, we'd loose output indexes (outs are referenced by 
txhash+index). We could use a sparse array for that, or move index numbers into 
TransactionOutputs and just search for the right index (could use binary 
search, although probably overkill).

Original issue reported on code.google.com by andreas....@gmail.com on 21 Apr 2013 at 10:59

[GoogleCodeExporter]

Yes, I was thinking this the other day.

Relaying transactions from the wallet is what allows you to create transactions 
offline or with flaky connectivity. We definitely don't want to lose that 
feature. However it only matters for pending transactions. Bitcoinj does not 
only create relevant outputs - obviously the output which goes to the other 
person is irrelevant to your wallet.

A tx is considered spent if all relevant outputs are spent. Irrelevant outputs 
don't change that, obviously, otherwise a tx sending money to someone else 
would never be considered spent.

I will eventually refactor the wallet to use much more compact internal 
representations when possible. I'll leave this bug open to track that.

Original comment by hearn@google.com on 22 Apr 2013 at 10:04

• Changed state: Accepted

[GoogleCodeExporter]

With the latest version, I've added a little bit more statistics. Here is a 
nice one:

Transactions: 88
Inputs: 90
Outputs: 78510 (spent: 24)

He tries to fit that into 32 MB of heap space...

Original comment by andreas....@gmail.com on 24 Apr 2013 at 9:04

[GoogleCodeExporter]

Here's a nice one:

Transactions: 78
Inputs: 274
Outputs: 130001 (spent: 90)

Excessive use of pay-to-many is on the rise. I hope we can accomodate for that, 
it's a valid usecase.

Original comment by andreas....@gmail.com on 7 Mar 2014 at 4:13