ethereum-optimism / ecosystem-contributions

Find ways to contribute to the Optimism Collective
MIT License
301 stars 115 forks source link

Foundation Mission Request: OP Stack Zero Knowledge Proof #61

Open bdresser opened 1 year ago

bdresser commented 1 year ago

Foundation Mission Request – OP Stack Zero Knowledge Proof

To take on this project, submit a proposal to this thread by June 28. Read more about Missions here.

How will this Foundation Mission (RFP) will help accomplish the above Intent?

Implementing a zero knowledge proof (ZKP) for OP Chains is a prerequisite to enable secure and low latency cross chain communication between L2 and L1, as well as directly between OP Chains. A ZKP for a well-supported instruction set architecture (ISA) that can prove the Optimism fault proof program acts as the foundation for a system that can prove any OP Stack based blockchain.

What is required to execute this Foundation Mission (RFP)?

Completing this mission means implementing a zero knowledge proof (ZKP) system that can prove the Optimism fault proof program using an instruction set architecture (ISA) supported by the golang compiler such as MIPS, RISC-V, or WASM. This proof system must additionally be shown to work in practice by proving the state transition between two blocks for a standard configuration OP Stack chain.

Support for the fault proof program introduces additional requirements on top of proving a standard execution trace for an ISA. Specifically, the fault proof program introduces the notion of the preimage oracle which uses special system calls to load external data into the program. Every fault proof vm is responsible for implementing some mechanism by which the hash of some data is placed at a particular location in memory, a syscall is executed, and the preimage for that hash is then loaded into memory for the program to utilize. The preimage oracle is also used to bootstrap the program with initial inputs. Please refer to the preimage oracle section of the fault proof program documentation for further information.

For this Mission, the Optimism Foundation will accept up to three submissions. Multiple parallel attempts at creating an OP Stack ZKP will increase chances of success and provide some friendly competition for building the best OP Stack ZKP.

Due to the wide design space, each solution will be evaluated based upon a few Solution Criteria. Due to the scope of this project, this RFP is not expected to be completed within governance Season 4 (June through September). Specific timelines will be discussed with each team that makes a submission.

The target grant payout for each submission will be 250k OP. Teams should discuss the solution criteria below with the Foundation, and may arrive at a different mutually agreeable grant amount dependent on the scope of their work.

Solution Criteria:

  1. Performance — What is the compute cost to generate and verify the proof?
  2. Latency — How long does the proof take to generate?
  3. Complexity — How many lines of code is the prover / verifier?
  4. Robustness — How confident are we that there are no bugs?
  5. Security — What are the assumptions the system relies on for its security guarantees? (e.g. trusted setup)
  6. OP Stack Compatibility — Are there fundamental changes required to the OP Stack?
    1. ****Note 1****: It may be the case that small changes to the OP Stack are required to achieve an acceptable performance benchmark. One example of this includes introducing a ZKP-friendly batch submission accumulator to reduce the cost of proving transaction inclusion in the OP Chain. However, the fewer changes the better.
    2. ****Note 2:**** The current OP Stack state transition function is defined in the fault proof program. Changes are evaluated with respect to this state transition function.

Proposals should include their target architecture and explicitly call out any tradeoffs their approach may have. Multiple proposals which select different tradeoffs may be accepted as each one can have their uses and bring value to the ecosystem.

How should the Foundation measure progress towards this Foundation Mission (RFP)?

Progress can be measured based upon the completion of the following milestones:

  1. Specification: A specification for the ZKP design. Should include the criteria which the implementation will aim to satisfy.
  2. PoC Check-in: A demo of the ZKP being used to prove the OP Stack state transition. The specifics of the PoC are left up to the builder, but the idea is show off the viability of the concept.
  3. Executable demo proving the OP Stack state transition between two blocks: An implementation of the ZKP working end to end and integrated with a live OP Stack chain. This should include instructions which allow anyone to test the proving system themselves.

These milestones mark the completion of the proposal.

Once the proposal has been completed, a final major milestone is the adoption of the ZKP by Optimism Governance. At this point the ZKP may be voted in as a default option in the Superchain bridge, making it a core component of the OP Stack. This is outside of the scope of this proposal but is included here to illustrate the critical importance of this Mission and the massive impact these proofs may have.

How should RetroPGF badgeholders measure impact upon completion of this Foundation Mission (RFP)?

Application instructions

To apply for this RFP, please complete the form in the expandable section below and leave your response as a comment on this issue thread below. Submissions will be open until June 28, at which time the Foundation will review all submissions and select up to three individuals/teams to complete the work defined here.

Submission form _Copy the entire application below and leave a comment on this issue with your answers completed. A representative from the Optimism Foundation may reach out using the contact info provided to request more information as necessary._ ## Foundation Mission (RFP) Application **Please verify that you meet the qualifications for submitting at the above [Tier](https://gov.optimism.io/t/collective-trust-tiers/5877/2)** * **Alliance Lead:** Please specify the best point of contact for your team * **Contact info:** * **L2 recipient address:** * **Please list the members of your Alliance and link to any previous work:** Read more about Alliances [here](https://gov.optimism.io/t/season-4-alliance-guide/5873) --- **What makes your Alliance best-suited to execute this Mission?** - [...] - [...] **Please describe your proposed solution based on the above Solution Criteria (if applicable):** - [...] - [...] **Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each peice of work:** - [...] - [...] **Please define the [critical milestone(s)](https://gov.optimism.io/t/grant-policies/5833) that should be used to determine whether you’ve executed on this proposal:** - [...] - [...] **Please list any additional support your team would require to execute this mission (financial, technical, etc.):** - [...] - [...] **Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant:** _(Note: there is no guarantee that approved Missions will receive up-front cash grants.)_ - [...] Please check the following to make sure you understand the terms of the Optimism Foundation RFP program: - [ ] I understand my grant for completing this RFP will be locked for one year from the date of proposal acceptance. - [ ] I understand that I will be required to provide additional KYC information to the Optimism Foundation to receive this grant - [ ] I understand my locked grant may be clawed back for failure to execute on critical milestones, as outlined in the [Operating Manual](https://github.com/ethereum-optimism/OPerating-manual/blob/main/manual.md#valid-proposal-types) - [ ] I confirm that I have read and understand the [grant policies](https://gov.optimism.io/t/token-house-grant-policies/5833) - [ ] I understand that I will be expected to following the public grant reporting requirements outlined [here](https://gov.optimism.io/t/suggested-public-reporting-requirements-for-grantees/4176) -- end of application -- ---
wangtsiao commented 1 year ago

I plan to give my zk solution and apply the foundation, but I am Ember Tier, am I eligible to apply? Because I see the Proposal Tier is Phoenix, which people generally don't have.

Love the OP's zk plan! But would you lower the proposal tier, so that more developers familiar with zk can join and build?

GrapeBaBa commented 1 year ago

I plan to give my zk solution and apply the foundation, but I am Ember Tier, am I eligible to apply? Because I see the Proposal Tier is Phoenix, which people generally don't have.

Love the OP's zk plan! But would you lower the proposal tier, so that more developers familiar with zk can join and build?

@wangtsiao If you try this one, I am glad to join if you are not care.

bdresser commented 1 year ago

@wangtsiao @GrapeBaBa apologies, this should have been listed as Fledgling tier per the descriptions here – I have updated the original RFP to clarify.

If you find that an RFP you are interested in is listed at a Tier that you haven't reached yet, you can:

wangtsiao commented 1 year ago

@bdresser Thanks for your clarification, love this RFP! BUIDL!

ashutosh887 commented 1 year ago

@bdresser I would like to work on this!

bdresser commented 1 year ago

cross-posting a proposal for this RFP posted by @wangtsiao on the governance forum

bdresser commented 1 year ago

@bdresser I would like to work on this!

Glad to hear it @ashutosh887 ! I'd suggest posting in the #alliances channel on the Optimism Discord, or reaching out to other teams submitting proposals to see if anyone is looking for additional help.

Abuchtela commented 1 year ago

I would love to help work on this

Cofson commented 1 year ago

Submitted one more proposal - thanks for posting this RFP @bdresser !

O1Partners commented 1 year ago

Foundation Mission (RFP) Application

Please verify that you meet the qualifications for submitting at the above Tier

Our ZKP-related work includes:

Please describe your proposed solution based on the above Solution Criteria

Solution Overview (definition of some key terms are set out at the end of this section)

Our proposed solution is to implement a fault-proof-program-compatible MIPS VM with efficient instruction execution embedded inside a Nova-folding-style bn128-kzg-plonkish system with deep Keccak support for efficient pre-image oracle access.

This would be used to prove arbitrary fault proof program execution traces. We believe we found a good point in the tradeoff-space that maximizes both OP stack compatibility and performance.

The proof system would be Kimchi, a Halo2-like PLONKish system currently configured with an inner-product-argument style polynomial commitment scheme. Kimchi powers the Mina Protocol. It supports verifiable computation using traditional turing-machine-based instruction sets.

Kimchi is designed to have a swappable backend and while the current implementation is defined over Pasta curves using an inner-product-argument-based polynomial commitment scheme (Pasta-IPA), the proposed implementation would swap in the KZG commitment scheme using the bn128 curves (bn128-KZG).

Solution Overview

MIPS_zkVM_(1)

Solution Flow

The proposer feeds the inputs to the fault proof MIPS binary which the bn128-kzg Kimchi system can interpret and ZK-prove the execution trace. Just as before, pre-image syscalls are resolved using the unmodified OP Stack pre-image oracle system. Finally, the proof gets sent to a smart contract on Ethereum L1 and then, if valid, updates state on the L1.

Components

Out of scope

Definitions

Criteria

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each piece of work:

We have grouped the major elements of work into Milestone phases, and detailed the elements of each phase in the table below, with time estimates attached.

Grant 1

Workstream Expected length of each Phase
Phase 1 / Milestone 1 - Complete and confirm specification with OP Labs: Finalized written specification 4 weeks
Phase 2 / Milestone 2: Demo of the ZKP being used to prove the OP Stack state transition: Low-level optimized Keccak in Kimchi, prover hashing changes, Keccak sys-call interface, Folding, Kzg-bn128 abstractions, VM instruction selection, and the complete VM ISA, instrument for the state transition flow. 10 weeks
Phase 3 / Milestone 3: An implementation of the ZKP working end to end and integrated with a live OP Stack chain , CLI host / pre-image integration, Documentation 2 weeks

Future Grants / Phases

Not included in the scope of this work but listed below for context and future consideration are some additional potential phases of work to complete and optimize availability of the solution within OP Stack:

Please define the critical milestone(s) that should be used to determine whether you’ve executed on this proposal:

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant: (Note: there is no guarantee that approved Missions will receive up-front cash grants.)

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

qizhou commented 1 year ago

Foundation Mission (RFP) Application

Please verify that you meet the qualifications for submitting at the above Tier

What makes your Alliance best-suited to execute this Mission?

Please describe your proposed solution based on the above Solution Criteria (if applicable):

Solution Criteria:

  1. Performance — Current state-of-the-art zkWASM prover at https://github.com/DelphinusLab/zkWasm needs a machine with 32+C, 1+TB memory, and 2x4090 GPUs
  2. Latency — Depending on the steps to prove, the time will range from a few minutes (~1K steps) to hours (~100K steps).
  3. Complexity — The zkWASM prover/verifier is already open-source at https://github.com/DelphinusLab/zkWasm. It supports BN128 curve and solidity verifier. Additional host circuits need to be implemented to support preimage oracles/hints.
  4. Robustness — DelphinusLab zkWASM is perhaps the most widely adopted zkWASM with a couple of near-production applications.
  5. Securityhttps://github.com/DelphinusLab/zkWasm is based on PSE Halo2 with needs of KZG ceremony (can re-use existing ones).
  6. OP Stack Compatibility — The op-program-client needs to be modified to support WASM target (e.g., fastcache is updated to support GOOS=js). No further modification is required in our current estimation.

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each piece of work:

  1. we will come up with the specification of the proposal solution in WASM ~ 2-4 weeks
  2. we will modify op-program-client so that the code can be compiled in WASM ~ 4-6 weeks
  3. we will build a zkWASM prover/verifier that can verified one and multiple steps on-chain ~ 8-16 weeks
  4. we will build a WASM emulator to emulate the execution of op-program-client ~ 4-6 weeks
  5. we will integrate all components to an executable demo ~ 2-4 weeks

Please define the critical milestone(s) that should be used to determine whether you’ve executed on this proposal:

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant: (Note: there is no guarantee that approved Missions will receive up-front cash grants.)

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

-- end of application --

stevenli27 commented 1 year ago

Foundation Mission (RFP) Application

Please verify that you meet the qualifications for submitting at the above [Tier]

Read more about Alliances [here]


What makes your Alliance best-suited to execute this Mission?

Please describe your proposed solution based on the above Solution Criteria (if applicable):

Our proposed solution architecture achieves the goals set forth by the RFP and also:

Please outline your step-by-step plan to execute this Mission, including expected deadlines to complete each piece of work:

To ensure project success, we propose an execution timeline spanning 6 calendar months in two parallel swim lanes:

ZK client

****L1->L2 derivation system (to be built concurrently with the ZK client)****

Epoch union (to be built after the L1->L2 derivation system)

Please define the [critical milestone(s)] that should be used to determine whether you’ve executed on this proposal:

Please list any additional support your team would require to execute this mission (financial, technical, etc.):

Grants are awarded in OP, locked for one year. Please let us know if access to upfront capital is a barrier to completing your Mission and you would like to be considered for a small upfront cash grant: (Note: there is no guarantee that approved Missions will receive up-front cash grants.)

Please check the following to make sure you understand the terms of the Optimism Foundation RFP program:

bdresser commented 1 year ago

Submissions for this RFP are now closed. Thanks to everyone who submitted a proposal!

Someone from the Optimism Foundation will reach out on or shortly after July 13 to communicate which proposal(s) have been accepted and schedule a kickoff.

In the meantime, feel free to tag me here or reach out directly (bobby@optimism.io) with any questions.

🚀

bdresser commented 1 year ago

Hi all – we are extending the selection deadline to July 20 to allow time for review. The Optimism team may be in touch to discuss the proposals above. Again, feel free to reach out with questions.

bdresser commented 1 year ago

Hi folks – thank you again for the excellent proposals here.

We're excited to share that we will be moving forward with the proposals from O(1) Labs and RISC Zero.

@stevenli27 and @O1Partners we will be in touch directly to kick off the project and discuss expectations in more detail.

For all other teams here: we'd love to help you find the right way to contribute to the Optimism Collective. See the rest of the issues in the Ecosystem Contributions repo for ideas, suggestions, and other possible projects. And stay tuned for more RFPs posted in the next few months. Thank you again for your proposals.

stevenpack commented 1 year ago

Hi OP community! Happy to share that work ⚒️ on this has kicked off at O(1) Labs, with detailed design 📋 and planning underway this week on optimized low-level keccak hashing and folding support in our Kimchi proof system. I'll share more details as we progress! 🏃‍♀️

stevenpack commented 1 year ago

Hi again OP community! 👋 Some updates on our work to provide a zero knowledge proof of the OP fault proof program 💻 . As a reminder, the major pieces of work (which can you view in our response) are:

You can checkout the low-level, optimized Keccak RFC here. Final comments almost resolved and implementation will kick off soon.

The folding implementation is broken up to 3 separate sub-streams.

Generalize exp framework - RFC approved. ✅ Code implemented and in review! ✔️ Kimchi quadricization - RFC approved ✅ Implementation ready to start 🚗 Folding - RFC approved ✅ Implementation depends on ☝️

Next up is the keccak-syscall integrations and CLI host integration.

Onwards! 🏃

intoverflow commented 1 year ago

Greetings Op community! 👋

The RISC Zero team is pleased to announce that we have made significant progress on the following deliverables:

For the block builder, we have added support for Optimism blocks to Zeth! With this update, Zeth is able to provably construct blocks from Op mainnet. We have tested against a handful of real-world blocks. For example, the proof for block 107728767 takes just under 1B cycles of execution. Using the Bonsai proof cluster (with 64 GPUs) we can prove this block in roughly 23 minutes.

For L1 -> L2 derivation, we have a branch in Zeth that demonstrates the ability to provably read from the Op sequencer using data from the L1. This branch has been tested on real-world data. We are currently optimizing and scaling the implementation.

As next steps, we will continue to expand test coverage for the block builder and improve the L1 -> L2 derivation system to handle large volumes of L1 data.

We are also coordinating with the zkVM platform team, who is working to add support for recursive proof verification. As their work completes we will begin building the “epoch union” tool, which will tie together the block builder & L1 -> L2 derivation system, and also allow proving sequences of Op blocks back to the L1.

qizhou commented 1 year ago

Hi, OP community👋!

I'm excited to present our collaborative efforts from the EthStorage team, in conjunction with Hyper Oracle and Delphinus Lab, in our pursuit of achieving Zero-Knowledge Fraud Proof using Wasm. With minimum modifications to op-program-client and introducing zkGo - a minimum modified Go compiler to produce Wasm code compatible with ZK prover, we are able to compile op-program-client into ZK-compatible Wasm code and pass the zkWasm dry-run with the witness generated by op-program-host!

To be more specific, the following is the overview of the challenges we faced:

To overcome these challenges, we have devised the following solutions:

  1. Minimum Modifications to op-program-client: We made minimal modifications to op-program-client to enable successful Go Wasm compilation and reduce the memory footprint of op-program-client to fit into zkWasm capacity.
  2. zkGo with Minimum Modifications to Go v1.21.0: We introduced minimal alterations to Go v1.21.0 to facilitate the compilation of op-program-client (and, in most cases, other Go programs with minimum modifications) that can be directly supported by zkWasm.

Repo to Replay: https://github.com/ethstorage/optimism/tree/js-io/op-program#build-wasi-and-replay-without-op-host-program

The full details can be found at https://perfect-amphibian-929.notion.site/Advancing-Towards-ZF-Fraud-Proof-zkGo-Compiling-L2-Geth-into-ZK-Compatible-Wasm-315a878af5754c549e5003568e1ee124?pvs=4

stevenpack commented 1 year ago

Hi again OP Community! 👋

Some incremental and positive updates this week. See our proposal above to get a sense of the overall workstreams.

Now back to work 👨‍💻

stevenpack commented 11 months ago

Hi OP folks and everyone interested in zero-knowledge proofs of OP! 👋

As always, see our proposal above to get a sense of the overall architecture and workstreams. As a reminder some of the big areas are: optimized keccak, kzg-bn128 implementation, MIPS ISA instruction processing, folding, integration with cannon and the pre-oracle hash.

Given the heavy use of keccak in EVM-based systems, its implementation in a zkVM needs to be efficient and so we've invested significantly in that area:

Recent updates:

Keccak's RFC has been updated and is available for review, the implementation is also ready for review. (I.e. initial PRs, not merged yet pending review)

Folding is still in progress and going well.

Our next intermediate goal is and end-to-end execution (with or without folding TBD), including interacting with cannon.

To that end, we've spent a lot of time getting to know cannon, the op-program, op-program-client and setting up test environments to run all these components together. Here is the RFC for this important workstream. Implementation work is about to start.


+-----------------+           +----------------+         +----------------+
|                 |           |                |         |                |
|                 |           |                |         |                |
|     Client      o<--------->o     zkVM       o<------->o      Host      |
|     (MIPS)      |           |                |         |  (op-program)  |
|                 |           |                |         |                |
+-----------------+           +----------------+         +----------------+

Seeing each workstream mature and the integration steps for an end-to-end demo kicking off is super exciting! ❗

See you in a couple of weeks!

stevenpack commented 11 months ago

Howdy OP-nation 👋 . Some great progress over here at O(1) Labs in the last 2 weeks. 🔥

  1. Folding landed in our Kimchi proof system -- it is being integrated into zkVM now. I love how succinct this API is.

image

image

  1. We're loading the op-program! That means we can read and parse all the instructions exported by Cannon. We're not successfully executing it e2e yet, so there are some instructions in the ISA we're debugging, but this is a good place to be.

  2. As part of that execution, we need to support syscalls for loading block data from the preimage. Check out the very detailed design here

Our next goals 🎯 are:

Onwards! 🏃‍♀️ 🏃‍♂️

stevenpack commented 10 months ago

No updates last 2 weeks due to team offsite and DevConnect. Regular programming (🤣) will resume shortly.

AustinBaggio commented 9 months ago

Hi OP & friends! I'm Austin, the new PM working with the Crypto team at O(1). We've been heads down on the PoC and recently hit an exciting milestone - the zkVM is fully compatible with Cannon!

The e2e demo, performance numbers, and docs to run it yourself are coming soon. Stay tuned!

intoverflow commented 9 months ago

Hi friends! Quick update from the RISC Zero team:

We've made some big improvements to op-derive (our implementation of L1 -> L2 Derivation). We've tested it on 10,000 real-world blocks from Op mainnet 🥳 The implementation will land into Zeth soon.

Our approach to derivation works like so:

The tool provides the following output:

For example, suppose we start from op block 110800000 and derive 3 blocks. The tool would output the following:

Eth tail: 18341377 0x4eb45301f17a8fcdbf65dd1122361db575792b517287121675bf856dfb8122f2
Op Head: 110800000 0x1545e3edbd5b7a5cb1ce231c9eabb6da6dd97596b8db21b81201187b5de6245e
Derived: 110800001 0x45eb3f54dfe4dfe462ebb5f13a4e48b4e3332f0ef48106f241f190dde7b68086
Derived: 110800002 0x89d059ee39f9d41560dbe7f658bb311628e04f7453b566aac5e48b9d01f5183a
Derived: 110800003 0x39c420afa9c7e36a5d2c90c1606035c5264fc3831e21b0fd9e2ae7c75d5e48fb

When we run this process in the zkVM, we obtain a ZK proof of the following:

If the Op head is in the Op history, and if the Eth tail is in the Eth history, then:

We have begun work on a final integration tool that uses recursive ZK proof verification (aka "proof composition") to put these pieces together. We hope to be testing the integration in January.

Until then, we wanted to share some updated performance data for zeth and op-derive.

For these tests, I used zeth to prove a sequence of Op mainnet blocks. (These proofs were generated using Bonsai, our proof service.)

Op block no Segment count Start time Stop time Elapsed
110800001 360 0:19:52 0:22:56 0:03:04
110800002        
110800003 509 0:38:54 0:42:45 0:03:51
110800004 101 0:49:24 0:51:11 0:01:47
110800005 860 0:59:05 1:05:12 0:06:07
110800006 602 1:05:28 1:09:50 0:04:22
110800007 118 1:10:15 1:12:03 0:01:48
110800008 262 1:12:28 1:15:18 0:02:50
110800009 1142 1:15:34 1:22:58 0:07:24
110800010 664 1:26:08 1:30:44 0:04:36

In the table above, segments is a measure of proof complexity (each segment represents roughly 1M cycles of execution). Time is denoted by hh:mm:ss.

We see that a typical Op block takes 2-8 minutes to prove. The exception is block 110800002, which for some reason takes much longer than the others (roughly 10k segments).

The proof times for op-derive are similar:

Op head Block count Eth head Eth tail Eth blocks read Segment count Start time Stop time Elapsed
110800000 10 18341358 18341377 19 1268 1:31:12 1:38:51 0:07:39
110800000 100 18341358 18341391 33 2077 1:39:26 1:51:57 0:12:31
110801000 10 18341523 18341541 18 1136 1:52:39 1:59:47 0:07:08
110801000 100 18341523 18341553 30 1916 2:00:16 2:11:45 0:11:29

Here I ran 4 jobs, two of them deriving only 10 blocks, and the other two deriving 100 blocks. These took roughly 8 and 13 minutes, respectively.

It's interesting to note that it's most efficient to derive large numbers of blocks at the same time. This is due to a combination of factors. In particular, the work required to perform derivation scales with the number of L1 blocks that need to be processed, and it is common for batcher transactions to be "lumped together." This means most Eth blocks contain little information relevant to derivation, but some Eth blocks contain enough information to derive several Op blocks. Larger derive jobs are able to take advantage of this, while smaller jobs end up doing a bunch of work only to throw most of it away.

Anyway! We're pretty excited to share this progress and these initial performance figures. In our next update, expect to hear more about how we're integrating op-derive with zeth to build an end-to-end proof 😁

AustinBaggio commented 9 months ago

Hey OP & friends!

Big update from us at o1 to close out 2023 - We have a working demo of the MIPS zkVM proving OP blocks! 🎉

Danny recorded a video demo, or you can run it yourself by following the commands in this readme. When you run run-code.sh, it will fetch the latest block for Cannon, run Cannon & the op-program to verify the claim is correct, run the zkVM, create a proof, and verify it.

To add a bit of context to the video (Updated): 0:00 - 1': Environment set-up. 1' to 3': Cannon runs with the op-program stopping at 100k instruction as opposed to executing the whole program to save time. There is a longer trace later in the video covering 100M instructions. 3' to 4'02: The zkVM is compiled and run on the same input as Cannon. Here we're showing that Cannon and the zkVM prove the same state transition successfully. 5:10: Shows proving 100M instructions. We log every 10M instructions and take a snapshot of the state. The zkVM log replicates the logs from Cannon. 27:15: The result of the 100M instructions.

Still to go:

The o1 team is off on holiday next week for a much-needed break. 🎄 See you all in 2024!

qizhou commented 8 months ago

Hello, OP community! 👋

Wishing you all a Happy New Year! We're excited to bring you the latest developments from our team, EthStorage, in collaboration with the hardworking teams at Hyper Oracle and Delphinus Lab.

Our journey in the realm of zk Fraud Proof since the release of the zkGo compiler in September 2023 has been marked by significant strides. Here's a breakdown of our accomplishments across three key areas:

In all, we're thrilled to share our recent progress. Our next milestone involves integrating zkWasm with the op-client's Go implementation compiled by our zkGo compiler for end-to-end proving capabilities. Stay tuned for our next update🚀🚀🚀!

AustinBaggio commented 8 months ago

Hey OP and friends, a quick update from O1:

Next week we’ll be focusing on integration testing. Stay tuned 🔥

intoverflow commented 7 months ago

The RISC Zero team is back with another update!

We’re thrilled to announce that we’ve completed our integration work and have generated validity proofs (and verified them onchain!) for a variety of real-world Op epochs, as well as 1 proof that rolls up a sequence of 180 Op blocks into a single proof (!)

As described in our previous update, our tooling splits up the proving process into various subtasks, such as reading transactions from the Op sequencer and executing those transactions in order to construct the corresponding blocks. In our previous updates we have given performance figures for these individual subtasks. Now that we have integrated these components, we are pleased to share the performance for the entire ensemble.

Epochs

We picked 10 random epochs and generated proofs for each. The following table measures proving performance using the Bonsai proving service:

Op head (block number) Op blocks in epoch Op transactions in epoch Op gas in epoch ZK cycles (sum over all subtasks) ZK cycles per gas Proving time (sum over all subtasks) Proving time per Op transaction
113963903 6 43 31,318,286 4,973,133,256 158.79 44.43m 1.03m
114020225 6 66 17,593,272 6,125,891,706 348.19 45.96m 0.69m
114089340 11 149 67,908,211 12,264,792,370 180.60 162.66m 1.09m
114165247 6 70 76,354,707 8,138,846,749 106.59 88.86m 1.26m
114223488 6 76 32,366,635 5,420,867,580 167.48 61.34m 0.80m
114547269 6 35 34,833,946 5,195,351,593 149.14 92.15m 2.63m
114354386 6 47 32,964,680 4,931,978,261 149.61 63.39m 1.34m
114400259 24 195 159,052,117 23,721,178,029 149.14 252.61m 1.29m
114453440 6 43 42,705,954 8,269,166,215 193.63 102.87m 2.39m
114511643 6 38 26,039,828 7,518,243,384 288.72 116.21m 3.05m

Rolling up 180 blocks

To demonstrate the tooling's composability features, we have also constructed a validity proof for a sequence of 180 Op blocks:

Op head (block number) Op blocks Op transactions Op gas ZK cycles (sum over all subtasks) ZK cycles per gas Proving time (sum over all subtasks) Proving time per Op transaction
114228596 180 1541 883,744,542 189,559,578,906 214.49 27.05h 1.05m

Performance observations

There is considerable variability in the number of ZK cycles required to process an Op transaction. Using the data above, the average number of ZK cycles required per gas is 196.54, though there is a fair bit of variable due to misalignment between the gas schedule and the ZK-cost of various EVM instructions.

The best indicator of overall proof cost seems to be the number of transactions. Here, we see that a typical proof requires 1.15 minutes to prove per transaction included in the proof. But again, there is a fair bit of variance due to outlier transactions.

We believe the average and variance in these statistics will be reduced through circuit and library improvements currently planned for later this year.

Onchain verification

We have verified the proofs listed above using our onchain proof verifier (on Sepolia). For example, our 180-block rollup was verified in this transaction.

These proofs cost less than 250k gas to verify. In a fully integrated solution, the Op L1 contract could use this verifier to validate proofs generated by our tooling, in either a full-validity mode, or alternatively in a ZK-fault-proof mode. In the latter, the dispute protocol could be simplified such that the accuser simply provides a proof for the block (or epoch) in question, definitively establishing its correct value.

Conclusions

These data demonstrate the end-to-end flow described in our project proposal. It has been exciting to get deep into the Op architecture, expand the capabilities of the Zeth prover, and build tooling for the L1 -> L2 derivation procedure.

It has also been exciting to see the progress made by our colleagues at O1 and the collaboration between EthStorage, Hyper Oracle, and Delphinus Lab. The future of ZK+OP is clearly bright and we look forward to the next steps for this Foundation Mission 🚀

modewei commented 7 months ago

Very well done.

0x1cc commented 6 months ago

Hello, OP community! 👋

Presenting a different perspective on zk fraud proof! I'm eager to engage in a discussion and hear everyone's insights.

ZK Fraud Proof with ZK State Channel

TL;DR

Background

Current interactive fraud-proof challenge protocols such as Optimism fruad proof system uses

Proposal

zkChannel

Advantages

Security Concern

One potential concern with this proposal is the possibility of non-cooperation between challengers and defenders. To address this issue:

Conclusion

This proposal works like an auxiliary plugin for expediting dispute resolution. It does not compromise the security of the existing fraud-proof system but instead offers a cost-effective and swift solution for dispute resolution.

AustinBaggio commented 6 months ago

Folding Demo 👀

Hey team, quick update from the o1 side. We have a quick folding demo that Fabrizio has prepared. This demo shows a constraint system with 2 gates and a circuit with 2 rows that creates 2 witnesses, verifies them, folds them, and checks the folded witness.

We're heads down on integrating Keccak and IVC, an e2e demo coming soon!

opjulian commented 5 months ago

Hi @intoverflow ! Just reached out to your team via email regarding KYB for your teams project

qizhou commented 2 months ago

Hello, OP community! 👋

We're excited to bring you the latest developments from our team, EthStorage, in collaboration with the hardworking teams at Hyper Oracle and Delphinus Lab.

Now, we have a working demo of proving the op-client smoke test (~2 billion instructions with 10 steps of derivation). Through continuation, the traces are split into 1056 segments, each with 2 million Wasm instructions, and are proved with zkWasm. The generated proofs can be aggregated into a single proof by our continuation-batcher.

In the process, we addressed the following key issues:

Our next milestone involves optimizing the proving system and integrating end-to-end proving with OP fault proof.

Eikix commented 2 months ago

Hey guys;

I'd like to know the opinion of OP Labs with regards to the two different approaches taken by Risc-Zero vs. O1 & EthStorage teams. Namely, use ZK to prove OP blocks (make OP a ZK-EVM), or use ZK to prove op-program / fraud proof system (make the fraud proof system ZK).

It feels preferable in my opinion to simply prove OP blocks, and change the protocol that asserts integrity of the chain, rather than force the fraud proof system to become ZK, but I may be misguided.

This is my understanding, let me know if I am wrong in my base premise.

joshuacheong commented 2 months ago

Hey guys;

I'd like to know the opinion of OP Labs with regards to the two different approaches taken by Risc-Zero vs. O1 & EthStorage teams. Namely, use ZK to prove OP blocks (make OP a ZK-EVM), or use ZK to prove op-program / fraud proof system (make the fraud proof system ZK).

It feels preferable in my opinion to simply prove OP blocks, and change the protocol that asserts integrity of the chain, rather than force the fraud proof system to become ZK, but I may be misguided.

This is my understanding, let me know if I am wrong in my base premise.

Thanks, similar with @Eikix I am keen to understand the status of the grants from various parties and their associated findings to understand which direction is the OP Stack going in terms of the two approaches to prove blocks (ZK Validity proofs) to prove a fraud (ZK Fraud Proofs).

dongchangYoo commented 2 months ago

I have discovered that many discussions about projects integrating Op Stack and ZKP are taking place here. We are developing Kroma, an Ethereum L2 blockchain using Op Stack. Additionally, we have already applied ZKP to fault proof system and have been operating the mainnet since September 2023.

We are developing a new version of zkVM based fault proof system that adds rigorous verification between batched transactions and the transactions used for ZKP generation. The structure of Optimism's Batcher is maintained. Additionally, since there is no process of uploading PreImages to the EVM like Cannon, it is cost-efficient. We believe this idea will be ultimately conducive to transitioning to zkRollup built on OP stack.

The proposed method is explained here. To keep it concise and readable, I’ve omitted commonly known information. If there are anything for improvement or parts that are unclear, please feel free to ask!