Scalability Design Explorations around changing Payment Flows

[nathan-at-least]

Some thought have been floating around for a while on this topic, and I'm not sure there's any good write-up, so I'm filing this ticket as a schelling point for these topics.

The basic insight is to examine how changing the "payment flow" UX and protocol might enable better points in the privacy & scalability space, for example by having senders somehow deliver shielded payment information directly to recipients instead of posting everything on-chain.

Fundamental Requirements

Any shielded payment protocol must meet these requirements:

• One or more senders must generate a legitimate transaction to any number of recipients.
• Each recipient must:
  • Learn about the transfers to themselves (including enough information to control those funds),
  • Verify that the transfers indeed "exist" in an eventually consistent protocol-global scope.
  • Verify that the funds being transferred are not double spent in a real-time consistent protocol-global scope.

Trade-off Space

The design around these general requirements balances a variety of important goals:

Usability

Ideals:

• senders can post transfers and receivers can receive them (after safely verifying them) very quickly, even with many many currently active users.
• recipients don't need to be online to receive, and indeed, they might be receiving to an private key buried in a pirate chest in a hidden cay.
• fees are very low.
• users can recover from lost/dead wallets safely from offline seeds, social recovery, or other schemes.
• Privacy:
  • no party except senders, recipients discovers a transfer was made from the senders to the recipients, unless either sender or recipient explicitly discloses those facts.
  • no party except senders and recipients can learn they have any connection beyond both using Zcash (except for voluntary disclosures).

Practicalities:

• When there is congestion due to many concurrent users, each user understands what is happening and can make an informed choice between waiting or paying higher fees.

Scalability

Ideally as the number of users grows the requirements for globally verifying existence and double spend protection grow sublinearly. This supports the UX goal of lower fees for greater adoption.

Relatedly, the state necessary to permissionlessly introduce new infrastructure is low, so that new infrastructure can join easily, and the state necessary to continue operation grows slowly so that it's easy/cheap to operate the infrastructure.

Data Availability

Ideally:

• when users drop phones in lakes, they can still recover funds from their safely backed up secret seeds.
• anyone can join the network infrastructure permissionlessly and retrieve all the necessary state to operate that infrastructure. We want to prevent data availability silos where a party could benefit from keeping their state proprietarily private. For example, suppose we introduced "double spend prevention proving services" to lower the state/cost of running validating nodes which can rely on proofs from these services, then those services might silo their data or withhold it from new entrants.

How Zcash Meets These Requirements Today

The first step of generating a transaction is local to a sender's wallet.

All of the recipient requirements are achieved by posting a full encrypted transaction to the blockchain:

• Recipients learn about transfers by trial-decrypting all shielded transfers.
  • If recipients don't have the full blockchain, they use a service (like lightwalletd) which introduces efficiency/privacy trade-offs.
• Recipients verify existence from the blockchain as they do trial decryption.
• Recipients verify double-spend protection by tracking all nullifiers on the blockchain and rejecting duplicates.
  • For light client recipients, they typically rely on a third party service (like lightwalletd) to perform this protection for them.

Privacy for wallets with full state replicas is excellent, because there's no network distinction for receipients (except timing of receiving transaction data).

Data availability comes from monolithically storing everything on the blockchain so all replicas have everything necessary. Data availability for light clients is a challenge for efficiency and privacy.

Scalability is poor, because all replicas must replicate all data for all users.

Acknowledgements

HT to conversations w/ @ebfull, Zooko, @vbuterin, @tromer, @nuttycom, @daira, and many others on these kinds of ideas over the past few years.

[nathan-at-least]

As a starting point for exploring changes from the current Zcash architecture: what if senders somehow deliver most transaction data directly to recipients, with only minimal commitment and nullifier state stored / validated globally?

This impacts trade-offs roughly like this:

• It loses some important data availability guarantees: if the recipients lose their data, they can't recover it (unless the senders resend it?)
• It gains scalability by minimizing the amount of global data. For example, users now could send very large "private memos" since those no longer are globally stored.
• It loses offline reception.
• It degrades privacy if it's possible to observe the transmission from sender to recipient.
• It may introduce consistency failures: what if the txn is committed, but the recipient never receives it or forgets it?

Maybe we can address some of the shortcomings in other ways? Perhaps a mixnet can protect network privacy and a "private mailbox" can provide data availability and allow offline reception.