Funding with small value inputs

[dev-warrior777]

Funding with small value inputs

Pool miners accumulate many small value mining share utxos which they may wish to trade. Mixing also tends to accumulate many small value utxos in a similar way. While this is maybe rare on average it is a potential problem for Dex transaction sizes. This may also lead in turn to large message sizes when sending those serialized transactions across the network.

Mitigation strategies:

1. Education:

• Ask pool miners not to use Dex wallets as an endpoint for mining rewards. Rather use another wallet and manually consolidate the many small utxos into larger utxos by sending to self, possibly repeatedly in batches. The same awareness could be extended to users of mixing services. Use a dedicated wallet for trading with less extreme utxo values.
• Document the above very simply in user-facing wiki.

  2. Early Warning:

• Identify code hotspots where this issue could surface
  • When funding new transactions
  • When serializing transactions for onward communication
• Add Logging at Warning level at such places
  • Initially use very conservative values
  • This logging would be visible for helping solve support issues
  • Logging can identify the problem scope

    3. Limit Inputs:

• Limit the accepted number of utxos used to fund a Dex transaction
• Limit the serialized size of a Dex transaction
• Error if the limits are exceeded

  4. Dex Consolidation:

• Proactively consolidate many smaller value utxos into larger ones in a similar process to the manual (1) above. Split transactions can be used.

These strategies may be seen as a simple progression. Education (1) is the first and simplest and awareness will lead to some mitigation of potential errors as seen by users. At the same time Early Warning (2) may be seen as a parallel step to see if there really is a problem, and the extent of that problem, especially 'in the wild'

If significant feedback from support issues and some development code testing warranted further action then step (3) Limit Inputs could be applied. The limiting of utxos consumed for funding is simpler. The limiting of serialized size is more akin to bitcoin code in wallet.cpp

If there are still frequent errors from either support or testing then the Dex could do it's own consolidation programmatically. This is the most complex mitigation strategy and should probably be considered only after steps 1,2,3 above have been seen as not enough to ensure error-free processing.

Proposed Now:

1. Simple user-facing wiki - can be very short
2. Early warning logging at conservative levels (based on feedback)
3. Monitoring

Related Issues:

https://github.com/decred/dcrdex/issues/1105

Feedback:

"Currently there's a client limit of 65k inputs, and for the comms, the payload is limited to 262144 bytes, which imposes a limit closer to 4k"

[jholdstock]

Pool miners accumulate many small value mining share utxos which they may wish to trade.

The set of users potentially affected by this issue extends beyond PoW pool operators.

VSP operators also collect a large number of very small UTXOs in the form of fee payments. I believe it could also affect heavy users of mixing because generally speaking, the more you mix, the more smaller UTXOs you end up with.

[chappjc]

Very thorough write up, thanks.

Regarding consolidation, that should be fairly straightforward. We have split transactions ingrained in our order pipeline, and the notion of a forced split is not new. It seems reasonable to me to re-do order funding if the first pass without a split ends up selecting more than a couple thousand inputs. The messy part will be aligning the "max order" code with that, but from a UX perspective doing these auto consolidations on the fly will be favorable to an error.

[dev-warrior777]

I still propose a phased approach to gather data on what may be happening. But I take your point because ease of use is a primary goal now.

[buck54321]

@chappjc was spot on. Consolidation functionality should be invisible to the user. Forced splits are the best solution, even if the algorithm is non-trivial. This issue is more important now that we're about to enable privacy for Decred. I'm not worried about VSP wallets. They can use a separate DEX wallet to solve the problem, and pointing DEX to their VSP wallet would be a weird config to begin with.