NU5 Tracking: The Spec

[dconnolly]

This epic will track the tasks that need to happen somewhere in code in order to implement Zcash Orchard and the other NU5 ZIPs in Zebra, as defined by the Zcash Specification.

This may include pieces of code contributed to the https://github.com/zcash/orchard crate or https://github.com/zcash/halo2 crate, by the zebrad team or the zcashd or whomever, but they need to happen by someone.

This tracking issue contains segments for the whole spec that includes Orchard changes for now. A checkbox means that that part of the specification describes something to implement.

A checked box or strikethrough means that we:

• have implemented that item
• are tracking that item in another Epic, or
• have otherwise resolved that item.

Our mandatory Canopy checkpoint significantly reduces the number of features that we have to implement.

Leaf nodes are marked by – and correspond to implementable items. They should be followed by issue references that track the status of that issue, or a short explanation. Some pieces of the spec that pertain to sending or receiving money or blockchain scanning are noted here, but not linked to issues, as we are not doing that work for Zebra for NU5 activation.

• ~1. Introduction~ – nothing implementable
• ~2. Notation~ – nothing implementable
• [x] 3. Concepts
  • [x] 3.1. Payment Addresses and Keys – #1864
  • [x] 3.2. Notes – #1864
  • [x] 3.2.1 Note Plaintexts and Memo Fields – #1864?
  • ~3.4. Transactions and Treestates~ (covered by #958 which is tracked under #1872)
  • [x] 3.7. Action Transfers and their Descriptions – #1864
  • [x] 3.8. Note Commitment Trees – #1864
  • [x] 3.9. Nullifier Sets – ~#1864~ #2230 #2231
• [x] 4. Abstract Protocol
  • [x] 4.1. Abstract Cryptographic Schemes
  • ~4.1.1. Hash Functions~ – pull as dependencies
  • ~4.1.2. Pseudo Random Functions~ – pull as dependencies
  • ~4.1.3. Pseudo Random Permutations~ - (will be done as part of the zebra-client work)
  • [x] 4.1.6. Key Derivation - #2044, #2080 (not required for NU5 validation)
  • [x] 4.1.7. Signature - #?
  • [x] 4.1.8. Commitment – #1864?
  • [x] 4.1.11. Group Hash – ?
  • ~4.1.13. Zero-Knowledge Proving System~
  • [x] 4.2. Key Components
  • [x] 4.2.3. Orchard Key Components – #1864?
  • ~4.4. Spend Descriptions~ (covered by ZIP-216/#1798 which is tracked under #1854)
  • ~4.5. Output Descriptions~ (covered by ZIP-216/#1798 which is tracked under #1854)
  • [x] 4.6. Action Descriptions – #1864?
  • 4.7. Sending Notes
  • ~4.7.2. Sending Notes (Sapling)~ - (will be done as part of the zebra-client work)
  • ~4.7.3. Sending Notes (Orchard)~ - (will be done as part of the zebra-client work)
  • ~4.8. Dummy Notes~ - probably doesn't exist in the wild yet
  • ~4.8.2. Dummy Notes (Sapling and Orchard)~
  • ~4.9. Merkle Path Validity~ (covered by #958 which is tracked under #1872)
  • ~4.10. SIGHASH Transaction Hashing~ (covered by ZIP-244 which is tracked under #1854 )
  • ~4.13. Balance and Binding Signature (Sapling)~ (covered by #1829 which is tracked under #1854)
  • [x] 4.14. Balance and Binding Signature (Orchard) - pool: #1895, #1980, #1939, #2102
  • ~4.15. Spend Authorization Signature (Sapling and Orchard)~ - (will be done as part of the zebra-client work)
  • [x] 4.16 Note Commitments and Nullifiers – ~#1911~ #2230 #2231
  • [x] 4.17. Zk-SNARK Statements
  • [x] 4.17.4. Action Statement (Orchard) – #1912
  • ~4.19. In-band secret distribution (Sapling and Orchard)~ (will be done as part of the zebra-client work)
  • ~4.19.1. Encryption (Sapling and Orchard) - #2041~ (will be done as part of the zebra-client work)
  • ~4.19.2. Decryption using an Incoming Viewing Key (Sapling and Orchard) ~ #2231
  • ~4.19.3. Decryption using a Full Viewing Key (Sapling and Orchard)~
  • ~4.21. Block Chain Scanning (Sapling and Orchard)~ nothing to do here?
• [x] 5. Concrete Protocol
  • 5.3. Constants
  • [x] 5.4. Concrete Cryptographic Schemes
  • [x] 5.4.1. Hash Functions
    • [x] 5.4.1.3. Merkle Tree Hash Function – #1864
    • [x] 5.4.1.6. DiversifyHash Hash Function – #1864 ?
    • [x] 5.4.1.9. Sinsemilla Hash Function –#1864, #2079
    • ~5.4.1.10. PoseidonHash Function - #2064~ - (will be done as part of the zebra-client work)
  • [x] 5.4.2. Pseudo Random Functions - #1864?
  • ~5.4.3. Authenticated One-Time Symmetric Encryption~
  • [x] 5.4.4. Key Agreement and Derivation
    • [x] 5.4.4.5. Orchard Key Agreement - #1864? (pull in generic reddsa?)
  • [x] 5.4.6. RedDSA, RedJubjub, and RedPallas – #1864?
    • [x] 5.4.6.1. Spend Authorization Signature - #?
    • [x] 5.4.6.2. Binding Signature - #?
  • [x] 5.4.7. Commitment Schemes
    • [x] 5.4.7.3. Homomorphic Pedersen Commitments – #1864
    • [x] 5.4.7.4. Sinsemilla commitments - #1864, #2079
  • [x] 5.4.8. Represented Groups and Pairings
    • [x] 5.4.8.6. Pallas and Vesta - #1864
    • [x] 5.4.8.7 Hash Extractor for Pallas - #1864 ?
    • [x] 5.4.8.8. Group Hash into Pallas and Vesta - #1864 ?
  • [x] 5.4.9. Zero-Knowledge Proving Systems
    • [x] 5.4.9.3. Halo 2 - #1864 (as a wrapping/parsing type) and also tracked under #1853 in #2104
  • [x] 5.5. Encodings of Note Plaintexts and Memo Fields – #1864
  • [x] 5.6 Encodings of Addresses and Keys - #1864
    • [x] 5.6.4 Unified and Orchard Encodings - #1864
      • ~5.6.4.1 Unified Payment Addresses~ - (will be done as part of the zebra-client work)
      • [x] 5.6.4.2 Orchard Raw Payment Addresses - #1864
      • [x] 5.6.4.3 Orchard Incoming Viewing Keys - #1864
      • [x] 5.6.4.4 Orchard Full Viewing Keys - #1864
      • [x] 5.6.4.5 Orchard Spending Keys - #1864
• ~6. Network Upgrades~ (tracked by #1854)
• [x] 7. Consensus Changes from Bitcoin
  • [x] 7.1. Transaction Encoding and Consensus #1917, #1980, #1981, #2103, #2105
  • v5 transactions are ZIP-225, which is covered by #1863 and tracked under #1854
  • [x] 7.3. Spend Description Encoding and Consensus - #1864 ?
  • [x] 7.4. Output Description Encoding and Consensus - #1864 ?
  • [x] 7.5. Action Description Encoding and Consensus - #1864 ?

Also remember that consensus ZIPs are part of the spec

[teor2345]

@dconnolly @mpguerra where's the ticket that tracks: "Pre-NU5 spec sections still active in Canopy or NU5, which Zebra hasn't implemented yet" ?

[mpguerra]

@dconnolly @mpguerra where's the ticket that tracks: "Pre-NU5 spec sections still active in Canopy or NU5, which Zebra hasn't implemented yet" ?

this is now #1872

[mpguerra]

@dconnolly I don't see any changes for NU5 in 5.4.1.5. CRHivk Hash Function, but I see them in 5.4.1.6 instead.

If that's correct, I will move the work in 5.4.1.5 into #1872 for the Pre-NU5 spec work

[mpguerra]

@dconnolly I started adding #1864 to some of the spec sections, can you please review these? Also, do we need to create an "Orchard Key Agreement and KDF" issue similar to #271 and/or an issue to "Implement Orchared shared secret key derivation" similar to #301 ?

[mpguerra]

For "4.1.6. Signature" do we need to create an issue for Pallas signatures?

[mpguerra]

For "4.1.10 Group Hash" are we going to pull in as a dependency?

[mpguerra]

For "4.7.2. Sending Notes (Sapling and Orchard)", do we need to create an new issue to implement Orchard note encryption and decryption similar to #181 and #269?

[dconnolly]

For "4.7.2. Sending Notes (Sapling and Orchard)", do we need to create an new issue to implement Orchard note encryption and decryption similar to #181 and #269?

We need to do this eventually for when we are sending money via Orchard but not for 'just' Zebra's NU5 chain validation. So yes we need to create the ticket but it does not have to be completed yet.

[dconnolly]

For "4.1.10 Group Hash" are we going to pull in as a dependency?

I think this is completed in #1864, under the hood it uses the pallas_curves dependency

[mpguerra]

Do we need to create an issue for " 4.9. Merkle Path Validity" also?

[dconnolly]

For "4.1.6. Signature" do we need to create an issue for Pallas signatures?

Yes I think so. I am stubbing out similar types to RedJubjub in that branch #1864 just to have the types, but I not sure yet if we just want to port redjubjub to Pallas, or great a generic RedDSA variant, or what, do I'm delaying the decision 🙈

[dconnolly]

@dconnolly I don't see any changes for NU5 in 5.4.1.5. CRHivk Hash Function, but I see them in 5.4.1.6 instead.

If that's correct, I will move the work in 5.4.1.5 into #1872 for the Pre-NU5 spec work

I think that is correct, yes

[str4d]

For "4.1.6. Signature" do we need to create an issue for Pallas signatures?

Yes I think so. I am stubbing out similar types to RedJubjub in that branch #1864 just to have the types, but I not sure yet if we just want to port redjubjub to Pallas, or great a generic RedDSA variant, or what, do I'm delaying the decision 🙈

As a reminder, I have already created a (semi-)generic reddsa from the redjubjub crate; it is what I'm using in the orchard crate. It's currently just a branch; I haven't published it yet because I want to discuss with you how you'd like to collaborate on maintenance going forward (since it's basically identical to redjubjub, and redjubjub could just be a wrapper around it).

[mpguerra]

Do we need to create an issue for "4.14. Balance and Binding Signature" ?

[mpguerra]

Do we need to create an issue for "5.4.9.3. Halo 2" similar to #305?

[teor2345]

Do we need to create an issue for "4.14. Balance and Binding Signature" ?

Pool balance is #1895, but we still need to create (or find) tickets for the rest.

[teor2345]

There are a significant number of transaction consensus rules in 7.1 that aren't covered by ZIP-225 or the v5 transaction RFC.

I'd like to track those changes in separate tickets, because the v5 transaction RFC is already quite large, and we need to finish it off soon.

[daira]

Note that some of the section numbers in the spec changed as a result of inserting section 4.1.3 Pseudo Random Permutations. Also 4.7.2 (sending notes) was split into separate Sapling and Orchard sections.

[mpguerra]

Updated section "5.6.4 Unified and Orchard Encodings" with "5.6.4.1 Unified Payment Addresses"

[mpguerra]

Added #2064 to section "5.4.1.10 PoseidonHash Function" and Epic

[mpguerra]

@dconnolly: Is "4.14. Balance and Binding Signature (Orchard)" fully covered by ZIP-209/#1895 ? If so, we can cross it out from this list

[mpguerra]

Looks like we are done here 🎉