Native SegWit v0 Offer Book

MaxHillebrand commented 4 years ago

JoinMarket started out before SegWit activation, and thus used to have only the legacy address format offer book. After the softfork activation, a new offer book was created, which uses P2PKH wrapped SegWit addresses. According to @AdamISZ, the reasoning was to preserve backwards compatibility, while still benefiting from the savings on blockspace usage. It seems that currently, the vast majority of offers are in the wrapped SegWit OB, and not in the legacy OB.

I open this issue to curate a conversation about starting a new native SegWit v0 offer book.

Pros

Even more blockspace savings compared to wrapped SegWit
Same address type as other useful privacy wallets, thus higher pre/post mix anonset [the two other frequently used CoinJoin wallets use only native SegWit, and all Lightning Network implementations too]

Cons

"Split" of the liquidity, similar to how it was with the move from legacy to wrapped OB.
Potentially a smaller anonymity set for early adopter users, especially those who exclusively use native SegWit

Alternatives

Instead of doing a third OB with native SegWit v0, wait until taproot is activated, and then create a OB with SegWit v1 directly, so to not need to split the liquidity again.

User feedback

@openoms has a twitter, mastodon, and telegram poll to gauge user demand.

AdamISZ commented 4 years ago

Another alternative is a mixed orderbook, though it superficially seems a bad idea, it may not be quite so simple, @chris-belcher has discussed this point in the past with us on IRC.

AdamISZ commented 4 years ago

It's a bit of a side-point but here is another reason to switch to bech32; BIP78 payjoin has just been merged in this repo but see this exchange with MrKukks (whose server I used to test):

<waxwing> Kukks : yeah that was nice :) (only after fixing bugs on regtest of course). one Q though: i used native segwit, would the same server also support p2sh-p2wpkh if I tried it?
<btcpay_bot> [mattermost - Kukks]ah nope,
<btcpay_bot> [mattermost - Kukks]the server only supports 1 format at a time for now
<btcpay_bot> [mattermost - Kukks]theoretically the protocol supports it and also supports switching its output to match your inputs/change output
<waxwing> right, you do not request `disableoutputsubstitution` and i do not set it. but if the server's wallet is native only, then it can't be done. so you would just fallback if i tried it with p2sh/p2wpkh inputs

If those more advanced merchants running servers are only offering payjoin for bech32 wallets, then it will not be very useful for current Joinmarket users (they can use bech32 wallets in Joinmarket, but they are not doing so, because the coinjoins are only for p2sh...).

openoms commented 4 years ago

It is time to tally these:

Overall there was an overwhelming majority in support of a Bech32 order book and no loud concerns.		yes
Twitter	116	14
Mastodon	7	1
Telegram	27	1
Tally	150	16
%	90	10

AdamISZ commented 4 years ago

There is the question whether a "mixed pit" is a viable idea, ping @chris-belcher for his thoughts on that. That idea would be letting addresses be anything. It is more complex, not only to reason about, but also to code - but not that much more complex.

There is the question of whether an upgrade to bech32 should form part of an overall Joinmarket messaging protocol upgrade - I had hoped this, and somewhat vaguely wondered whether there might be a jump to taproot/Schnorr at the same time. But of course I had no idea about taproot activation time and still don't.

In case you don't know what I mean my an upgrade to the messaging protocol, I mostly mean that there are several unoptimal encodings in the data passed across (things like very non-compact signature representations, and even one case of double-base64 encoding which is quite :man_facepalming: ). We could probably add fidelity bond options at the same time, and also make things like commitments more flexible.

But let's suppose we don't address that and just purely make new bots be bech32 pit only.

PRs welcome :) It would actually be a rather excellent way for someone to get their hands dirty and understand some details of Joinmarket's architecture.

If that was the decision, the list of actions to be taken are something like this:

jmdaemon.protocol.py module. This defines, as part of the protocol, the list of valid offer types. We need a new offer type that new bots understand and old bots ignore. I guess the rest is unchanged if we don't upgrade the protocol itself.
the default wallet type for all tools is now jmclient.wallet.SegwitLegacyWallet (which inherits from BIP49Wallet and has several mixins). But this is controlled by the user with the [POLICY]: {segwit, native} option settings (see jmclient.configure.py for details), so we would only need users to switch over to native wallets. Nothing in the wallet needs changing.
one technically tricky part: in the protocol the Maker passes their transaction signature to the Taker in a !sig message. The contents of that message will change from having three items to only 2, as should be obvious from considering how BIP141 defines the scriptSig and witness fields for p2sh-p2wpkh and p2wpkh respectively. See jmclient.maker.on_tx_received() for the construction of the sigmsg. Obviously there is the same issue to address in jmclient.taker.py.
that's ... kind of it :) I believe there is nothing in Joinmarket-Qt that needs changing, though I could be wrong. The biggest issue around this shift is that no approach is perfect for all users. Some just want to keep doing what they're doing and will resent the fact that liquidity is split (the first paragraph of this comment is relevant). Others will be impatient that we have not cajoled everyone to switch. The poll data is useful, and thanks for that, but only a little bit.

decentclock commented 4 years ago

Thank you for elucidating the possible paths forward.

But let's suppose we don't address that and just purely make new bots be bech32 pit only.

Question: under this option, would new users still have the option to make / take offers in nested segwit? Or would we force any new offers to be native segwit?

I'll let people with more experience decide on what option we end up going for. I do not have strong opinions on this for now.

PRs welcome :) It would actually be a rather excellent way for someone to get their hands dirty and understand some details of Joinmarket's architecture.

I have a ton of time on my hands until September, so I would be very happy to implement whatever option we end up going for. I think this is a very timely topic given the current status of the mempool, and we should make something happen.

MaxHillebrand commented 4 years ago

If there is one mixed orderbook, by default, which address type is used by the wallet?

ALWAYS segwit
ALWAYS wrapped segwit
randomly chosen with some weighted average

AdamISZ commented 4 years ago

If there is one mixed orderbook, by default, which address type is used by the wallet?

Well I was hoping @chris-belcher would chime in on this, but from what he said on IRC he doesn't have a strong opinion about this at all. Not sure it's worth bothering digging up the old discussions we had on this 6 months to a year ago on IRC. If he's not inclined in that direction, neither am I and so we can just forget it. I think we agree it's too complex to be very practical even if a case can be made.

That leaves us with going for bech32. There will be a liquidity split, which isn't without its drawbacks.

Question: under this option, would new users still have the option to make / take offers in nested segwit? Or would we force any new offers to be native segwit?

Well by definition they can't make two different offers with the same coins, right. As for 'force', I mean basically it is not possible in the sense that we cannot force people to run a certain version of software. And we don't have an intention to, even if we could; the most we can do is release a new version and strongly recommend upgrading to the new one. But due to the nature of such a marketplace, it inevitably is a slow process. Edit: There is a valid question I guess (this is probably what you meant) as to whether new versions will still support offers on p2sh pit if native=false is set in the config (and in the wallet, don't forget wallet types are fixed at generation time). I think we should continue to support both.

You'll see we had a short discussion about this on freenode (http://gnusha.org/joinmarket/2020-07-24.log starting 07:51)

takinbo commented 4 years ago

In case it needs to be restated, enabling a bech32 orderbook could save a ton on fees per input (25%). I'm concerned though if this would be future proof in supporting Taproot when (if?) it gets activated? If the same orderbook would accept p2tr inputs and/or outputs, then I suspect the upgrade will be made anyway and we could as well do so now. If it would, yet again, need another orderbook, should we consider delaying it?

decentclock commented 4 years ago

@takinbo I do not have deep knowledge of the codebase, and I am not a core maintainer here. But from what I can gauge from the community, the timeline for taproot is still not certain. So I do not think we should wait until then to get the fee-saving benefits you mention.

@AdamISZ @chris-belcher @kristapsk Do any of you have thoughts on whether we should wait for schnorr?

MaxHillebrand commented 4 years ago

there are two important timepoints regarding taproot, the first is activation of the softfork [I'll make a wild guess of 2-5 years from now] and then majority usecase on the network, compared to legacy and segwit v0 [that's another 2-5 years after activation].

So, even in the best case, it is years until many coins are using segwit v1.

Thus, it is no option for me to "wait till taproot, in two weeks!"

On a second note, I don't think it's a good idea to mix segwit v0 and v1 coins in one order book, they have different on-chain fingerprints and can thus reduce privacy and anonymity set. [depending on the implementation, of course]

decentclock commented 4 years ago

I'm getting started with the implementation to make this happen. I should have something by the end of this week.

AdamISZ commented 4 years ago

OK. Keep in touch on IRC if you can @jules23 , it'll help avoid losing time on stuff.

openoms commented 4 years ago

To help testing I will activate and keep online a native Segwit Yield Generator on testnet.

On the RaspiBlitz / JoininBox people can switch to testnet with SETTINGS -> [x] Testnetor connect to a Bitcoin Core on testnet. In the JoininBox menu I added a command option to update JoinMarket to a specific pull request: ~/install.joinmarket.sh testPR [PR number] For example to install with this PR about the Native SegWit orderbook: JoinMarket-Org/joinmarket-clientserver#656 can run: ~/install.joinmarket.sh testPR 656

Edit the joinmarket.cfg (CONFIG menu):

network = testnet

native = true

to make the next wallet generated a native segwit one

minimum_makers = 1

to be able to coordinate a 2 party coinjoin

openoms commented 4 years ago

Ready to be tested on mainnet:

While I'm doing this if anyone wants to sanity check by running a bot or two on mainnet that would be great :) https://github.com/JoinMarket-Org/joinmarket-clientserver/pull/656#issuecomment-724073233

For that in JoininBox /RaspiBlitz joinmarket user:

$ cd
$ ~/install.joinmarket.sh testPR 656

Edit the joinmarket.cfg (CONFIG menu):

network = mainnet

native = true

to make the next wallet generated a native segwit one

minimum_makers = 1

to be able to coordinate a coinjoin with only one maker.

Generate and fund a new wallet now with nativeSegWit (bc1...) addresses. Start the YG or the sendpayment.py (SEND).

In a new tmux window you can keep your old wallet running too using the command line.

MaxHillebrand commented 4 years ago

Closed by #656

JoinMarket-Org / joinmarket-clientserver