This is a meta-issue to capture and give context to various adapters we are supporting or want to support.
General problem: we want to make it as easy and flexible as possible for ZK app devs to use the fastest for their needs.
Background
We started off with Circom/Groth16 support through arkworks/circom-compat + wasmer for witness generation. This was much faster than snarkjs stack, and led us to be able to create a performant pocket-kimchi PoC with the old anon-aadhaar circuits.
As a PoC in a few hours we also added experimental support for Kimchi (Plonkish proving system) to show how easy it is to add more adapters (https://github.com/zkmopro/mopro/pull/34).
As anon-aadhaar and proof of passport matured, we started looking more closely at performance limitations of our current stack and tried to support these project on their journey towards production usage. However, we ran into unexpected problems when it came to app store submissions due to dylib/xcframework problems:
While in theory this is possible (Worldcoin launched app with it), we couldn't get it to work.
At the same time we tried circom-witness-rs (https://github.com/zkmopro/mopro/issues/32) which seems very promising. However, it has a bunch of features that are missing to get it to work for all the circuits we need it to work for.
Finally, we recently found a workaround by switching from wasmer to wasm3: https://github.com/zkmopro/mopro/pull/127 - this addresses the TestFlight hard blocker, but it is much slower than wasmer.
Due to this, we saw some teams moving away from current mopro stack to Rapidsnark+Witnesscalc stack (https://github.com/zkmopro/mopro/issues/44). The primary motivation for moving was (1) No TestFlight blocker (2) Faster witness generation (since it is native). There was not enough developer resources at the time to integrate this into Mopro more properly, also considering it is a different architecture vs Arkworks.
That's where we are at now.
Groth16 adapter roadmap
Rough sketch.
1) Merge wasm3 work. This means we have something that works, even if it is slow.
2) Get witness-rs to feature completion. I believe we aren't that far off with this with a lot of great work by Vivian and Chase. It'd fit very easily into current stack (we already have a feature toggle) and it'd allow us to be 100% native with very good performance.
3) Explore and integrate other native speedups. This would either be Rapidsnark+Witnesscalc and/or Tachyon. Based on current understanding of benchmarks, Tachyon seems more performant. Especially assuming we get witness-rs to work. This could entail Rapidsnark+Witnesscalc too, potentially, as it is quite mature. See section below for sketch on how to integrate this.
First step is getting it to work on mobile, this is issue 143. Second step after that is to integrate it with mopro. This would give us several advantages:
Uniform and easy access to multiple platforms (iOS/Android/RN/Desktop/CLI) with native language bindings.
Same DevEx in terms of mopro init/build/export-bindings etc
Allow app developers to write business logic in Rust and do minimal changes to go from, say, circom-compat to tachyon
Since Tachyon is written in C++ this requires a thin shim layer in Rust to interact with. Luckily, Rust was very much designed with this in mind, since C++ is dominant in systems programming and Rust was designed to improve on this. Unlike a lot of things we do in ZK-land, this is not a new problem. There are likely several mature approaches here, either something like rust-bindgen (with a minimal C wrapper for a simplified interface over C++ templates for generic functions) or https://cxx.rs.
By defining a thin Rust wrapper over this C++ code base (e.g. for witness generation and proving) we can more easily integrate this with mopro-core but also other consumers can use it in their Rust apps (a schelling point for many ZK apps and infra).
There's also a GPU exploration happening that can be viewed as its own adapter too, trying to leverage GPUs on mobile.
Others might come up, but this should be based on a strong need before we spread ourselves too thin. Right now, getting Groth16/Circom and Halo2 right would give us most bang for the buck. Later on, there are many different adapters that'd be interesting to support. As the project matures, we should also encourage contributors to submit their own and make it easy to add new ones in a modular fashion (a key design goal of mopro). See https://github.com/zkmopro/mopro/issues/16
Adapter system
Right now, the adapter system is fairly primitive. As we add more mature adapters, going to n=3 or so, we should better be able to see clearly what the right abstractions here are and avoid over-engineering for the time being.
For now, the general logic is to:
1) Have separate namespaces for each adapter
2) Leverage feature flags to only conditionally compile relevant adapters
3) Try to keep udl FFI interface somewhat the same, and either prefix with adapter name or use different files for this (conditional compilation)
4) Expose adapter in mopro-config.toml file and mopro-cli
5) Use this to create a custom template app with only the relevant code
Context
This is a meta-issue to capture and give context to various adapters we are supporting or want to support.
General problem: we want to make it as easy and flexible as possible for ZK app devs to use the fastest for their needs.
Background
We started off with Circom/Groth16 support through arkworks/circom-compat + wasmer for witness generation. This was much faster than snarkjs stack, and led us to be able to create a performant pocket-kimchi PoC with the old anon-aadhaar circuits.
As a PoC in a few hours we also added experimental support for Kimchi (Plonkish proving system) to show how easy it is to add more adapters (https://github.com/zkmopro/mopro/pull/34).
As anon-aadhaar and proof of passport matured, we started looking more closely at performance limitations of our current stack and tried to support these project on their journey towards production usage. However, we ran into unexpected problems when it came to app store submissions due to dylib/xcframework problems:
While in theory this is possible (Worldcoin launched app with it), we couldn't get it to work.
At the same time we tried circom-witness-rs (https://github.com/zkmopro/mopro/issues/32) which seems very promising. However, it has a bunch of features that are missing to get it to work for all the circuits we need it to work for.
Finally, we recently found a workaround by switching from wasmer to wasm3: https://github.com/zkmopro/mopro/pull/127 - this addresses the TestFlight hard blocker, but it is much slower than wasmer.
Due to this, we saw some teams moving away from current mopro stack to Rapidsnark+Witnesscalc stack (https://github.com/zkmopro/mopro/issues/44). The primary motivation for moving was (1) No TestFlight blocker (2) Faster witness generation (since it is native). There was not enough developer resources at the time to integrate this into Mopro more properly, also considering it is a different architecture vs Arkworks.
That's where we are at now.
Groth16 adapter roadmap
Rough sketch.
1) Merge wasm3 work. This means we have something that works, even if it is slow.
2) Get witness-rs to feature completion. I believe we aren't that far off with this with a lot of great work by Vivian and Chase. It'd fit very easily into current stack (we already have a feature toggle) and it'd allow us to be 100% native with very good performance.
3) Explore and integrate other native speedups. This would either be Rapidsnark+Witnesscalc and/or Tachyon. Based on current understanding of benchmarks, Tachyon seems more performant. Especially assuming we get witness-rs to work. This could entail Rapidsnark+Witnesscalc too, potentially, as it is quite mature. See section below for sketch on how to integrate this.
Sketch for integrating Tachyon
Related issues:
First step is getting it to work on mobile, this is issue 143. Second step after that is to integrate it with mopro. This would give us several advantages:
mopro init/build/export-bindingsetcSince Tachyon is written in C++ this requires a thin shim layer in Rust to interact with. Luckily, Rust was very much designed with this in mind, since C++ is dominant in systems programming and Rust was designed to improve on this. Unlike a lot of things we do in ZK-land, this is not a new problem. There are likely several mature approaches here, either something like rust-bindgen (with a minimal C wrapper for a simplified interface over C++ templates for generic functions) or https://cxx.rs.
By defining a thin Rust wrapper over this C++ code base (e.g. for witness generation and proving) we can more easily integrate this with mopro-core but also other consumers can use it in their Rust apps (a schelling point for many ZK apps and infra).
Other proving systems
Outside of Groth16/Circom there are also other adapters that are relevant. The most immediate one is Halo2: https://github.com/zkmopro/mopro/issues/145
There's also a GPU exploration happening that can be viewed as its own adapter too, trying to leverage GPUs on mobile.
Others might come up, but this should be based on a strong need before we spread ourselves too thin. Right now, getting Groth16/Circom and Halo2 right would give us most bang for the buck. Later on, there are many different adapters that'd be interesting to support. As the project matures, we should also encourage contributors to submit their own and make it easy to add new ones in a modular fashion (a key design goal of mopro). See https://github.com/zkmopro/mopro/issues/16
Adapter system
Right now, the adapter system is fairly primitive. As we add more mature adapters, going to n=3 or so, we should better be able to see clearly what the right abstractions here are and avoid over-engineering for the time being.
For now, the general logic is to:
1) Have separate namespaces for each adapter 2) Leverage feature flags to only conditionally compile relevant adapters 3) Try to keep
udlFFI interface somewhat the same, and either prefix with adapter name or use different files for this (conditional compilation) 4) Expose adapter in mopro-config.toml file and mopro-cli 5) Use this to create a custom template app with only the relevant codeNotes
Some recent benchmarks:
Questions and comments are welcome!