Improve transaction privacy / fungibility in Bitcoin Core and the Bitcoin system [meta tracking issues]

[gmaxwell]

Financial privacy is an essential property of every monetary and payment system. Without it the user's security is reduced due to targeted attacks, the freedom to associate and transact is limited, and their commercial interests may be compromised when competition can monitor their activity.

The importance of privacy was recognized in the original Bitcoin whitepaper, but the only approach available at the time (pseudonymous addresses) provides only limited privacy and what it does provide is highly brittle. Advanced monitoring capabilities have substantially undermined this protection in practice, but at the same time technology has improved tremendously and we can now do much better than pseudonymous addresses.

Tightly linked to privacy is fungibility, an essential characteristic of a money like good. When coins are overly distinguishable and people find themselves feeling obligated to consult (likely centralized) blacklists before accepting coins the utility of Bitcoin as a money is reduced.

Some people previously believed stronger privacy was incompatible with public auditability, but just like digital signatures prove your know your private key without disclosing it we know that privacy and public verification are not incompatible in a very fundamental sense.

Bitcoin Core should have a comprehensive plan to give users state of the art technology in this regard, but we must respect the challenges of deploying new technology in an already existing ecosystem. Progress can be made in a series of larger and smaller initiatives.

This issue with track other issues and collect discussion related to the overall strategy on this front.

Currently Open Issues (without pull requests):

• Implement Coinjoin in walltet. #3226
• Change value privacy #6569

Proposed short term functionality:

• Implement some kind of ephemeral (ECDH) address.
• Improve privacy in coin selection
• Include a private transaction submission tool enabled by default

Proposed longer term exploration:

• Merkelized abstract syntax trees or other explicit support for script elision in the spirit of coinswap (https://bitcointalk.org/index.php?topic=321228.0) to keep smart contract terms as private as possible
• Confidential transactions: What performance envelope, security considerations would we need from something like or zero cash to seriously consider it?

[gmaxwell]

ECDH addresses have been previously discussed in the Dark Wallet "stealth address" scheme. I'm not a super big fan of that scheme which has never been completely specified.

It results in addresses which are more than 2x larger than our current ones, largely so it can support a secure delegated scanning mode. But, of course, the scanning delegation totally undermines the privacy of the system. Adam Back and I spent some amount of time exploring fancier crypto to try to repair the scanning damage, but short of ought switching to pairing based signatures (which are slow to verify) we don't really have a way to fix the delegation.

But the ECDH code in libsecp256k1 can scan something like 40,000 transactions per second on a fast desktop. With that in mind, we could instead forget about delegation.

I was contemplating a scheme for a future elements update called "Donation addresses" which would give you a stealth like ephemeral address which is as short as (or shorter) than a current Bitcoin address. But you couldn't securely delegate the scanning: someone who could scan for your payments could spend the coins. This might be a worthwhile trade-off.

[rnicoll]

I'm not entirely clear what you mean by "private transaction submission tool", could you elaborate please?

Also as I suspect there will be those reading this who may not be aware of the confident transactions suggestion, link for their reference: https://people.xiph.org/~greg/confidential_values.txt

[gmaxwell]

There are more advanced cryptographic privacy schemes which have been proposed.

For example, there is my Confidential Transactions scheme (https://people.xiph.org/~greg/confidential_values.txt) which is running in elements alpha sidechain (https://github.com/ElementsProject/elements). In that sidechain CT is implemented as a major change to the transaction format, so it would be a hard fork to apply it directly.

But it's actually possible to implement CT as a soft fork, though ... more than a little kludge. Ignoring how ugly that might or might not be for a moment--

What performance envelope do we think needs to be achieved for something like this to deploy it in Bitcoin?

Maybe it would be helpful if I made a brief table that laid out the performance implications very clearly?

There are other schemes which could be considered, for example Zero Cash, but it's less clear how to soft fork deploy them (except e.g. via the extension block concept). Again, it's unclear how we can go about reasoning on the acceptable trade-offs in this space.

[gmaxwell]

@rnicoll 0.11 added a feature to suppress the local broadcast of transactions by the wallet. The idea is that a daemon could run along side your Bitcoin Core and submit your transactions via a high latency privacy network, or the like. If widely used this would greatly discourage the common practice of running spy nodes that suck up everyone's connection slots watching for TX broadcasts.

[sunnankar]

@gmaxwell These are very important issues at the network consensus level and is an issue of high priority at the enterprise and institutional level.

In the meantime, regarding the ability to avoid spy nodes there is already a process in place using features from other available wallets which shrinks the attack surface to something very manageable and controllable. (1) craft the offline transaction with Armory using a full Bitcoin Core node so there is no leakage of any information to a third-party when looking up address balances and (2) privately (TOR, etc.) submit the signed transaction through https://blockchain.info/pushtx (using the 'Copy Raw Tx HEX' feature in Armory).

[akstunt]

@sunnankar that is in armory and tor though. We need solutions for this in core not in armory or by use of Tor. It MUST be a prerogative that "normal" people aren't being spied on per-say.

[pierce403]

@gmaxwell If you haven't already, please take a look at how Vertcoin has implemented stealth addresses in their reference client. They have a "stealth addresses" tab with a "new" button on the bottom, along with a "import stealth address transactions" button. This eliminates the need to be constantly scanning for stealth transactions, so there is zero performance hit for non-stealth users. If you are using stealth addresses, you just hit the scan button from time to time when you're expecting something.

I agree that centralized delegation is no good, but I feel like people who would use stealth addresses would be just fine running a full node, or they would accept the security compromise that comes from verifying from an SPV wallet. Maybe if bitcoind allowed for some obelisk style verification commands, then SPV clients could ask the nodes themselves to scan as needed.

[Giszmo]

In this context the impact of hub and spoke networks on fungibility and vice versa the necessity of fungibility for an actually decentralized hub and spoke system should be considered.

Hubs need liquidity.

Anonymously run hubs might also function as mixing services for anonymous liquidity providers.

Policy makers might impose strict rules on hubs and consequently on users of non-complying hubs if lack of fungibility reveals their use. This could lead to the centralization many see as a danger with the lightning network.

If hubs work as mixes they will pay incentives resulting in zero or even slightly negative transaction fees.

[cjepson]

I think the ideal implementation of ECDH would be by offloading secret exchange to a peripheral network. Probably BitMessage at this point. It has multiple implementations and is probably one of the more supported networks for messaging. That said, if I'm remembering right it itself still requires the addition of ECDH to messaging for further privacy. I'm not sure that's a huge new feature to add.

You would send the information needed to redeem the transaction along with the transaction ID directly to the payee over this channel. It's simple and doesn't result in appending any information to transactions that could result in further censorship. It's possible that the addition of the OP_RETURN data itself could result in the closing of accounts at major exchanges, which would generally negate the usefulness of all of this. In this scheme, there is also no blockchain scanning because users communicate directly.

CoinJoin may be a dead end because of its voluntary nature and the fact that it will be so easily open to manipulation so long as the channels to enter a CoinJoin transaction are public. Programs with involuntary inclusion of inputs in spending seem superior (e.g. ring signatures with "key image" inclusion), although I doubt we'll see an inclusion of that with a Bitcoin soft fork.

[SergioDemianLerner]

I would like you to consider my cut-and-choose+random-subset-test method for creating range proofs when you think of adding confidential transactions to Bitcoin. I blame myself for not publishing this proof in a clear separate proposal for you to check. I will publish it separately soon. It has very similar properties as borromean signatures (a big larger proofs, I think). It has the benefit of being much simpler, and easily allows 64-bit of numeric ranges without doing floating point encoding.

[kristovatlas]

ECDH addresses have been previously discussed in the Dark Wallet "stealth address" scheme. I'm not a super big fan of that scheme which has never been completely specified.

It results in addresses which are more than 2x larger than our current ones, largely so it can support a secure delegated scanning mode. But, of course, the scanning delegation totally undermines the privacy of the system.

Have you reviewed BIP47 Reusable Payment Codes #159? By using notification transactions, clients do not need to outsource finding payment addresses to scanning delegates.

[adlai]

@cjepson CoinJoin is actively used by JoinMarket, which already supports sending directly from the core wallet. There are several issues preventing JoinMarket from inclusion in Bitcoin Core (such as the "pit" being an IRC channel, part of the project's relative youth), but it's worth noting that this approach is far from "dead end".

[cjepson]

@adlai The problem with CoinJoin is the difficulty in ensuring most or all of the participants are not malicious. If you have 4 inputs and 3 of them are an attacker in a CoinJoin, then the attacker knows which input and output you own. This gives the illusion of privacy, which may be worse. As with all of the input mixing methods, including ring signatures with ZKPs to prevent double spending, there is the possibility of sybil attack which makes them less attractive as privacy solutions.

[adlai]

The problems with privacy tools are of the "not a bug; feature!" variety, as they dispel the illusion of magic invisibility cloaks, revealing instead a market for anonymity set membership. (This discussion may be more appropriately continued in #3226)

[kristovatlas]

@cjepson I don't think that criticism is still relevant. Chaumian blinding in theory can help "blind" CoinJoin. But in fact, CoinShuffle already exists for this purpose, and is simple to implement.

Edit: Oops, I didn't read the post correctly.

[leo-bogert]

I've got some more privacy-related issues for you: "Clients leak IPs if they are recipients of a transaction": https://github.com/bitcoin/bitcoin/issues/3828 "Implement nested multi-paper backup" (the "nested" is the privacy, or rather stealth, part) https://github.com/bitcoin/bitcoin/issues/3212

[Midar]

Another interesting way to achieve privacy is using ring signatures like in the CryptoNote protocol (I'm surprised this hasn't been mentioned already).

[cjepson]

@kristovatlas CoinShuffle doesn't solve the Sybil attack as far as I'm aware. If you own all the other outputs of the CoinShuffle except for that of a single external participant, you still know that the remaining output you don't own belongs to the single participant.

Sybil attacks on input mixing were investigated classically by Monero: https://lab.getmonero.org/pubs/MRL-0001.pdf

I'm not sure how Chaumian style blinding helps either, but maybe I haven't thought hard enough about it. It just lets a third party sign for spending the output without knowing what the output is.

[cjepson]

@Midar I mentioned it above. Voluntary versus involuntary input mixing.

[kristovatlas]

@cjepson: You're right, my bad. I didn't read your post correctly. Sybil attacks are hard. Joinmarket potentially places incentives to reduce Sybil attackers overall % of the network, but I haven't seen any economic analysis of that yet.

[SpaceBitz]

ECDH as i understand it have a very similar result as generating unique addresses for every receiving payment, i don't see how it is any privacy improvement. When you spend the coins received by the ECDH method you still have the same problems of today, the recipient knows where the coins came from etc.

For the short term something already known and recognized should start being implemented as of today, so many years have passed and the promised privacy of bitcoin is still a myth. Built-in CoinJoin in the Core wallet should be good enough for now while something better is studied for the long term, but should be enabled by default in the wallet to avoid discrimination against its users if the mehtod happens to be detectable in the blockchain.

[otheATgh]

No it´s not. CoinJoin/Shuffle whatever - all of those interactive solutions are a pain for the users to use, its not enough to put them into BTC CORE. For obvious reasons, most people don´t even use BTC Core they are on Electrum, Multibit, Copay and whatever else wallet; most of them prolly use their mobile phone etc. "Premixing" coins just to be able to be a bit anonymous? Total No-go from a user experience perspective. Bitcoin needs to get more simple in every way to be successfull, not more complicated.

If you want to archive better privacy in BTC, then go with a protocol level change and enforce it for everyone, always.

This is all cool for us nerds but definately not for my Mom if we ever want an average person to get involved into Btc.

[palexande]

thoatbk's comments are correct. Unless the innovation is included in the protocol and can thus be used at a full node, it is less useful for decentralized and distributed cryptocurrency computing. (Which is why Cryptonote/Monero is a strong contender in the arena of cryptocurrencies.)

[theuni]

@gmaxwell Do you have a POC for CT as a soft-fork? Or maybe a high-level description for what would be necessary? I'd be interested to have "kludge" qualified a bit.

[cjepson]

@theuni The major issue I would say is that transactions spending multiple masked amounts now have a minimum size of 2,564 bytes for the range proof alone. Bitcoin is struggling with transaction volume of smaller transactions. I'm not sure it's softfork implementable as a new OP_CODE where you can pay out to masked outputs. What the amount/value field in the TX out would be considered is unclear. Normally it has to sum to <= inputs. If you zero/mask them out you'd need to add new consensus rules about using the outputs elsewhere in the blockchain as inputs. That's hardfork. I'm not sure it's super easy to drop it in. But maybe someone has some better ideas.

On the other hand, stealth TX negotiated off chain add no new bloat to the Bitcoin blockchain and appear as normal transactions where address reuse prevention is enforced. I think this should get a priority, as it requires no modifications to Bitcoin.

[chris-belcher]

@cjepson there is no cryptosystem in the world that can give you transaction privacy if all the users except you are sybils who co-operate with each other.

Even if you used zerocoin but every single other user was the NSA, you would still not get privacy from them.

So yes your criticism of CoinJoin is true but it's also true for everything else. Of course in some systems it might be easier or harder to get the required numbers to sybil attack.

[eragmus]

A bitcoin cloud mining firm with 5 petahash capacity partners with an AML firm to "monitor and report suspicious activities":

http://www.prnewswire.com/news-releases/bitcoin-and-blockchain-focused-hashingspace-retains-services-of-identitymind-global-an-anti-money-laundering-and-regulatory-compliance-platform-300134571.html

HashingSpace Corporation, a bitcoin ASIC mining and hosting company, announced today their decision to use the IdentityMind platform to monitor and report suspicious activities, as an integral part of the firm's commitment to compliance with all regulatory requirements worldwide.

IdentityMind Global™ is recognized as the leading provider of anti-money laundering and compliance services for Bitcoin and virtual currency companies across 4 continents, and offers the most advanced compliance platform for virtual currencies. The company provides a full suite of risk monitoring and Anti-Money Laundering technologies, enabling businesses to comply with US (FinCEN) and worldwide regulations.

[anirgu]

Hashing Space smells like a total scam. Just judging by the amount of "no news" press releases they have issued over the past few months. I would stay away IMHO

On Mon, Aug 31, 2015 at 1:56 PM, eragmus notifications@github.com wrote:

http://www.prnewswire.com/news-releases/bitcoin-and-blockchain-focused-hashingspace-retains-services-of-identitymind-global-an-anti-money-laundering-and-regulatory-compliance-platform-300134571.html

HashingSpace Corporation (OTCQB: HSHS), a bitcoin ASIC mining and hosting company, announced today their decision to use the IdentityMind platform to monitor and report suspicious activities, as an integral part of the firm's commitment to compliance with all regulatory requirements worldwide.

IdentityMind Global™ is recognized as the leading provider of anti-money laundering and compliance services for Bitcoin and virtual currency companies across 4 continents, and offers the most advanced compliance platform for virtual currencies. The company provides a full suite of risk monitoring and Anti-Money Laundering technologies, enabling businesses to comply with US (FinCEN) and worldwide regulations.

— Reply to this email directly or view it on GitHub https://github.com/bitcoin/bitcoin/issues/6568#issuecomment-136496809.

Amith ฿. Nirgunarthy | IHB https://ihb.io/ +1.310.795.9666 | +1.201.839.6148 (USA) | +91.9611.415.486 (India) Angel https://angel.co/amith-nirgunarthy | LinkedIn https://www.linkedin.com/in/anirgu | @anirgu https://twitter.com/anirgu | Facebook https://www.facebook.com/anirgu | Google+ https://plus.google.com/+AmithNirgunarthy/about | Skype: anirgu

[erik777]

Break it down into the known risks as axioms, then present infinity type assumptions to them to describe the "best" solution in a non-infinity world.

Core Axioms

1. The Internet is based on IP communications.
2. Your direct peers know your IP.
3. Your outputs will be recorded in a public ledger.
4. Your inputs will be recorded in a ledger.
5. Various transaction in the ledger are associated with identities by third-parties.

Second Level Axioms (Networking)

1. The more hops between you and the reader of the ledger, the less chance there is of associating your inputs/outputs with your IP. (Infinite hops eliminates the association.)
2. Encryption between hops can help protect from network snooping.

Third Level Axioms (Transactions)

1. Increased participants in a transaction reduces individual identity. (Infinite participants in a transaction eliminates identity.)
2. Increased anonymity between participants in a transaction decreases identity.
3. Increased cost of transaction (net loss) decreases risk of information gathering only participants.
4. Increased spending stake decreases risk of information gathering only participants.
5. Increased profit stake increases risk of information gathering only participants, and can negate #4.
6. The primary privacy goal is to eliminate the association of the input Bitcoin address from the output address.

Fourth Level Axioms (Randomization Through Pooling)

1. Increased pools size increase individual identity outside pool.
2. Increased hops to pool increases identity within pool.
3. Hashes and public keys can decrease identity within pool.
4. Increasing pool size can increase transaction latency.
5. The pool needs a means of output the transactions in a way that minimizes exposure of input identity.
6. Minimizing viewers of the building of the pool minimizes risk of those outside it identifying participants.
7. Destruction of the pool after it completes its mission minimizes leakage of its internal history. Absolute destruction may not be obtainable, but this is an avenue that could be explored to at least minimize persistence of its past.
8. Separating the input from the output increases privacy.

Looking at this, it appears like a side-chain can be used to build a pool. However, obtaining a desirable size can result in significant latency. Perhaps one inevitable trade-off is increased latency for increased privacy.

The trick is to create a pool protocol with the following goals:

1. Individual nodes, especially non-participants in a pool, have minimal knowledge of each pool while still facilitating its building.
2. Building of the pool is completely decentralized.
3. Have algorithms that match participants to create a complete pool.
4. Have the ability to prove to Bitcoin that the final transaction is authentic.
5. Allows the creator(s) of the final transaction to do so without being participants in it.

Example Use Case Joe wants to send 1 BTC to Sally. He specifies that the pool size must include no less than 100 participants, and each participant must have a stake of no less than 0.1 BTC.

Joe authorizes the side-chain to withdraw 1 BTC from Bitcoin while securing the output within the side-chain. Only the node who can execute the final pooled transaction with BTC can know the recipient, and that node will by then have no way of knowing it was from Joe. The only known by intermediary nodes is the amount of the transaction. The encrypted output is "authenticated" but unknown.

This permits the authenticated output to become part of the temporary ledge as a candidate for a pool. When enough candidates build, and perhaps other "proof of work" criteria was met, the pool transaction outputs could be sent to Bitcoin. At this point, no details of the pool are persisted within the side-chain.

Challenges

1. Cryptography must hide the destination address from all but the last node.
2. The side-chain must be trust-less, yet be able to act on behalf of Joe with his permission, particularly with regard to withdrawing his BTC.
3. The side-chain must be able to protect Alice's BTC after that point.
4. The final node transferring Alice's BTC must be able to do so, yet without the ability to steal it.
5. The final node should not require any knowledge of how the pool was built. It just knows the requirements were completed.
6. There needs to be a way to ensure that this does not result in duplicate Bitcion transactions. How will the input and output transactions be created on Bitcoin?

This last challenge appears to require that side-chains become a feature of Bitcoins. This could eliminate the concept of a final node completely, as this side-chain could reach consensus and simply "broadcast" the final transaction to Bitcoin. That way, 100 nodes could do this, and it wouldn't matter.

Integrating with Bitcoin Whether this pooling became a part of the Bitcoin is irrelevant because no matter what, the building of a pool requires a side-chain. It makes sense to just put general side-chain capability in Bitcoin, and just consider this a use-case.

Bitcoin can define this as a collateral side-chain, "locking" withdrawals until outputs are made, maintaining a side-chain balance. This would mean that Joe's 1 BTC would be locked, perhaps with an expiration. Alternately, a side-chain could "deposit" with Bitcoin collateral equal to the maximum withdrawal balance. A hybrid of these two could be possible as well. Regardless, to build a pool totalling 100 BTC these funds would have to either be provided as collateral to Bitcoin first, or temporarily "lock" withdrawals (tx inputs) until the pool dispersed its funds back to Bitcoin (tx outputs). Ideally, many pools would build simultaneously to separate withdrawals from deposits, though. So, we could realistically be talking about 1000 BTC to support 10x100 BTC pools building at once. If it could only build one at a time, then inputs could be associated with output pool.

Side chains can charge TX fees, which can then be used to recoup any collateral, or free up any locked funds. Competition between side chains would keep these fees low.

In any event, I don't see this type of side-chain happening without a well designed fork of Bitcoin to enable it. I also, given the axioms, don't see this type of privacy being possible without a hard fork.

Ironically, I didn't imagine this requiring side chains when I started writing this. This is the first time I really thought it out, as well. Now that I have, I think I really love them. Despite the accusations of them favoring Bitstream, if designed right, they will be equally beneficial to EVERYONE. In this scheme, it is entirely decentralized and trust-less. Anyone can fire up a side-chain and potentially profit from it. Anyone can use any side chain open to the public, such as the privacy one I just described. Public ones will compete, creating an open marketplace. And, as I described, the side-chain itself can choose to be completely decentralized and trust-less. Nodes will have to run it, and will require an incentive, possibly earning transaction fees. But, that's no different than Bitcoin today.

[nopara73]

I would like to bump this issue up by pointing out that, since 2015, the Bitcoin on-chain fees have grown and considering current trends, it is safe to assume they will continue to grow until an unexpected breakthrough happens.

This brings up a new issue regarding works on privacy schemes: today, privacy schemes must consider high fees. For example a privacy solution that eats block space may not be the good way to go, this has became painfully apparent our with TumbleBit's Classic Tumbler mode implementation.
On the flip side Sybil attacks and some DoS attacks became less of an issue, there are more room to play with, since participation in privacy schemes are becoming increasingly expensive.

[Sjors]

@gmaxwell should Dandelion be on your list above (e.g. under "Include a private transaction submission tool enabled by default")?

[Rspigler]

~The gold standard for privacy and fungibility in Bitcoin at the moment is @nopara73 's project Wasabi wallet. Since 1.0 was just released, creating an issue here now to implement the ZeroLink Fungibility Framework (https://github.com/bitcoin/bitcoin/issues/14622) as a part of this meta tracking issue.~

Edit: removed due to centralization concerns. Possible future Byzantine Cycle Mode

[nopara73]

Wasabi's mixing is centralized, so it is not a good fit for Core.

[eloyesp]

I would mention here #19035 that I've submitted in the same line, the idea is that coin-selection could be based on the idea of coin cluster (not just coin source).

[ilanoh]

We often use the metaphore "tainted" to describe privacy leaks. If you push this idea to its limit. Think about what happen when mixing different colors of painting again and again. You always end up with black paint. There was a study that demonstrated that 90% of usd paper bills are tainted with cocaine. So what we stop using them because at some point in the bill history someone took a hit with this bill? According to AML rules yes. But in the end the reality is that actors that would enforce this rule will be self driven out of the market. In our case with bitcoin, using coins for decades will taint them all to absolute black like color mixing paint. Which will naturally drive off the market regulated centralized actors that have to follow the AML rules, as they won't be allowed to accept any coins anymore. Thus creating a demand for decentralized actors immune to AML rules by essence. This flippening will take time, but will eventually occur, as tech develop.

[pinheadmz]

Privacy and fungibility are crucial, ongoing goals of the project. This meta tracking issue has gotten stale and I think it is safe to close. Almost all the recommendations in the OP have been developed or are in-progress such as silent payments, address groupings, and Taproot. 🚀

[adlai]

@pinheadmz could you please edit your closing comment to include links, reducing the amount of fingerprinting that must be suffered by folks reading along?

[pinheadmz]

@adlai https://en.bitcoin.it/wiki/Privacy

[adlai]

@adlai https://en.bitcoin.it/wiki/Privacy

Why should I donate my metaphorical guano to the bitcoin-doteyetee domain?