use minimum input for shuffle amount (wishlist)

[markblundeberg]

In relation to #67 , in every shuffle it's possible for there to be at least one player who doesn't need a change output, while at the same time preserving full anonymity. The way this would be accomplished is to increase the shuffle output amount to be equal to the minimum input, minus fee.

Doing this will help reduce overall amount of shuffles, as for every round, 1 of the players will 'get lucky' and not have to do more shuffles on the change output. If the poolsize is 10 then for every shuffle you have a 10% chance of not needing to do more shuffles.

[cculianu]

Hmm. You know this is not that hard to do .. since I'll be changing the code that crafts tx's anyway...

[Mengerian]

Yeah, I endorse this idea.

It has many advantages. The main one is that it should result in far fewer small dusty outputs of 0.001 or 0.0001 BCH, since each shuffles has a reasonable probability of being changeless.

I think it should be added to the roadmap, with a target for being the default mode of operation when other wallets like Bitcoin.com implement the protocol.

[cculianu]

I agree with both of you to the Nth degree. 1000%. Jonald says he's amenable to it (yay) so maybe a description or ELI5 about how this is the same and how this is different from what he have now might go a long way?

[cculianu]

(basically any lobbying for this is good!)

[Mengerian]

OK, I can try to write something.

The caveat is that I'm not super-familiar with the protocol details. For example, it will need a new protocol version for communication, since the final transaction is constructed in a different way.. But I don't know if/how protocol version is handled

[markblundeberg]

ELI5: You have 4 players at the 1.0 BCH tier:

• Player A brings 4.2 BCH
• Player B brings 3.9 BCH
• Player C brings 9.9 BCH
• Player D brings 2.2 BCH

(Players cannot bring less than 1.0 BCH at this tier, otherwise they would have to play at a lower tier.)

Instead of creating a transaction that outputs 4 x 1.0 BCH shuffled coins (and four change coins), you can do a transaction that outputs 4 x 2.2 BCH (and three change coins).

[cculianu]

The caveat is that I'm not super-familiar with the protocol details.

Ha yeah. :) I am still learning the protocol myself as I didn't write the code from it and it evolved a little bit since the spec was written. Basically on the protocol (as in on the wire) level it's not invasive at all. In fact it could be done as a 20 line change probably to all of the code.

All participants know this about each other:

1. their input utxos,
2. the output shuffled and
3. output change addresses.

They also know: "we all agreed to shuffle amount X". (where now X is one of discrete values: 0.0001, 0.001, 0.01, 0.1, 1.0, 10.0).

What they do: They go about independently signing the same canonical TX they all imagine the other people agree to, and pass signatures around for that tx.

So -- here we are just redefining what is that canonical TX. The new canonical TX is just a shuffle amount based on the smallest guy in the pool, rather than a fixed amount for that tier. As @markblundeberg said.

That's is. :)

Code-wise not very invasive on the protocol level.

[Mengerian]

@cculianu Is there some place where the current "canonical" transaction construction rules are written?

I'm thinking about the details like what amounts are deducted for fees, and the rules around not having dust outputs, with the amounts going to fees.

All that stuff should stay basically the same... Just the "shuffled output" amounts being changed. And the fact that it will always have a maximum of N-1 change outputs (fewer if people have similar input amounts within dust limits)

[imaginaryusername]

Additionally, I think this can have its own branch, and merged with main branch soon after release when we have sufficiently tested new logic, to minimize disruption of the security audit.

Possible logic regarding to tiers:

I propose that the "tier" system be retained; you still join pools appropriate for your amount as you do now (You do not join a 0.1 BCH pool if you have 1.1 BCH output, for example). However, after communication rounds finished in everyone knowing all the inputs of a tx, there can be a round where the shuffled output amounts is bumped to the lowest input amount (as @Mengerian posted). This might be an additional round of communication as participants adjust their amounts.

Note that this will likely require change to the shuffled-output-identification logic. It can no longer identify by flat tier amounts, but instead must identify by "having >3 outputs of identical amounts, one of which is mine".

Upgrade path:

This will likely be incompatible and require a third shuffle type at server (first being default/flat rate, second being dustshuffle). For maximum liquidity and compatibility, I'll recommend that wallets first attempt to join Type 3 pools, then have a long timeout (hours?) after which they bump themselves to attempt Type 1 pools; after a shorter timeout (say, 1 hour) they bump themselves back to Type 3.

Alternatively they can stay in Type 3 all the time, and only check stat port of server to opportunistically join almost-complete Type 1 pools. Please comment on what you think is better.

[Mengerian]

there can be a round where the shuffled output amounts is bumped to the lowest input amount

It shouldn't require any communication. It just needs a "canonical" rule that everyone applies in the same way.

And yeah, you definitely need to keep the tiers. In fact, doing this method may reduce the need for many tiers, so just sticking to power of 10 should be fine.

Note that this will likely require change to the shuffled-output-identification logic. It can no longer identify by flat tier amounts, but instead must identify by "having >3 outputs of identical amounts, one of which is mine".

Yeah, this is important also, if that' not the current logic.

[cculianu]

@cculianu Is there some place where the current "canonical" transaction construction rules are written?

@Mengerian -- yes. I'm glad you asked! This surprisingly tiny function in the python: https://github.com/clifordsymack/Electron-Cash/blob/b3164a40c099021cf3713685e45578aed530e309/plugins/shuffle/coin.py#L86

Each client runs this function against basically the same set of inputs to generate the same tx. They then proceed to exchange signatures for that tx in a secure manner.

All that stuff should stay basically the same... Just the "shuffled output" amounts being changed. And the fact that it will always have a maximum of N-1 change outputs (fewer if people have similar input amounts within dust limits)

Yes. Bingo! Ding ding ding. It's trivially easy to make that amount be dynamic. I literally would be surprised if it's more than a 10-line change. It's like a "consensus rule" -- get it right early and then the upgrade path is easier later. Now's the time to do this because the inertia later will be too great.

[cculianu]

@imaginaryusername

Note that this will likely require change to the shuffled-output-identification logic. It can no longer identify by flat tier amounts, but instead must identify by "having >3 outputs of identical amounts, one of which is mine".

@Mengerian

Yeah, this is important also, if that' not the current logic.

Yes. I looked at the "identify as shuffled" code and it uses rules based on tx shape -- it should still be ok -- it uses a heuristic/tx shape method to identify the tx as "shuffled" and it restricts it to the discrete amount bins. Loosening the criterion to "any amount, so long as it looks right" shouldn't produce too many "false positives" in the spirit of "if it looks like a duck, quacks like a duck -- it's a duck". If a tx looks shuffled .. it is shuffled. :)

[cculianu]

For the curious this simple loop decides "shuffled?" and it doesn't appear to be a big deal to loosen the restriction to be any amount, so long as it has the right tx shape: https://github.com/clifordsymack/Electron-Cash/blob/b3164a40c099021cf3713685e45578aed530e309/plugins/shuffle/qt.py#L73

[Mengerian]

Yes. I looked at the "identify as shuffled" code and it uses rules based on tx shape -- it should still be ok -- it uses a heuristic/tx shape method to identify the tx as "shuffled" and it restricts it to the discrete amount bins. Loosening the criterion to "any amount, so long as it looks right" shouldn't produce too many "false positives" in the spirit of "if it looks like a duck, quacks like a duck -- it's a duck". If a tx looks shuffled .. it is shuffled. :)

From the perspective of someone doing blockchain analysis, it's just the multiple inputs of the same amount that matter, not whether it's an "even number". So I can't see why you'd need to retain an "even amount" check.

[markblundeberg]

This surprisingly tiny function in the python:

From what I recall, that function is supposed to sort outputs by amount ... and clearly this isn't happening here.

Never mind. I see it's different from the one in wallet.py

[cculianu]

Never mind. I see it's different from the one in wallet.py

There's one in wallet.py?! That one might be dead code.. looking now. :P

[cculianu]

Can you show me what you were looking at in wallet.py @markblundeberg ? It is probably dead code and should be removed..I can't find it...

[markblundeberg]

Can you show me what you were looking at in wallet.py @markblundeberg ? It is probably dead code and should be removed..I can't find it...

I mean in lib/wallet.py there is the "real" make_unsigned_transaction that gets used for all normal transactions.

[cculianu]

I mean in lib/wallet.py there is the "real" make_unsigned_transaction that gets used for all normal transactions.

Yeah that's just regular.. non-shuffle. :) phew you scared me for a second. It's not used while shuffling but is used to craft tx's outside of shuffling.

Note that when spending the decision about which coins to spend happens before that function is called in what is basically the "send" tab ... :)

[zquestz]

I support making this change. I think it should be done sooner rather than later too. This is one that significantly reduces the number of utxos created, is a simple change, and it is better to make this breaking change now than have another shuffle type.

[imaginaryusername]

@zquestz do you mean we should implement this asap as a type 1 shuffle breaking change, and eliminate "upgrade" considerations?

I'm fine with that if the timeline allows and it won't break audit too badly - up to people writing code.

[cculianu]

Hopefully won't break audit. It doesn't change much about the crypto stuff -- just the layer on top that talks about the tx. 🤞

I'm going to implement this as a fork we can test.. it's not even that invasive. With any luck in a week's or two week's time we'll all be like "why did we ever do fixed amounts anyway?"

It's not a huge change.. thankfully!! 😃

[zquestz]

Yes I think this should be fixed before we release. That way we don't need to worry about how to upgrade and people don't get blamed non-stop for slightly different clients.

[cculianu]

This is now committed to this branch: https://github.com/clifordsymack/Electron-Cash/tree/shuffle_lowest_amount

It works!

I urge you guys to try it out. Don't worry -- you won't interfere with the legacy shufflers as this client now advertises itself as "version=200" which means you won't be matched with the older clients in the pools.

Here's an example shuffle I did on testnet:

https://explorer.bitcoin.com/tbch/tx/91a9adad5522c3a0f83b113d1b16c44bf8dcacac724b26fc2ee0d808888455e6

[cculianu]

I will wait and may merge this to master tomorrow. Just going to run through the changes again with a fine-toothed comb...

[cculianu]

PR here: #87