Open p4u opened 2 weeks ago
This is super cool!
I see that some of the public inputs are zero. For efficiency, we by default use the in-complete formulas when computing a multi-scalar multiplication and zero as a public input would hit the edge case.
To force using complete formulas, you can provide the groth16.WithCompleteArithmetic()
verification option. In your example you should do:
@@ -416,5 +430,5 @@ func (c *VerifyCircomProofCircuit) Define(api frontend.API) error {
if err != nil {
return fmt.Errorf("new verifier: %w", err)
}
- return verifier.AssertProof(c.VerifyingKey, c.Proof, c.PublicInputs)
+ return verifier.AssertProof(c.VerifyingKey, c.Proof, c.PublicInputs, stdgroth16.WithCompleteArithmetic())
}
By the way - we also have the "test engine" which does not create a proof, but instead evaluates the circuit using *big.Int
and checks the in-circuit assertions. This is allows for faster debugging of the circuit without needing to set up the keys etc. For example:
@@ -13,6 +13,7 @@ import (
"github.com/consensys/gnark/frontend"
"github.com/consensys/gnark/frontend/cs/r1cs"
"github.com/consensys/gnark/logger"
+ "github.com/consensys/gnark/test"
"github.com/rs/zerolog"
"github.com/consensys/gnark/std/algebra/emulated/sw_bn254"
@@ -229,6 +230,19 @@ func main() {
var pk groth16.ProvingKey
var vk groth16.VerifyingKey
+ // Create the circuit assignment with actual values
+ circuitAssignment := &VerifyCircomProofCircuit{
+ Proof: recursionProof,
+ VerifyingKey: recursionVk,
+ PublicInputs: recursionPublicInputs,
+ }
+
+ err = test.IsSolved(placeholderCircuit, circuitAssignment, ecc.BN254.ScalarField())
+ if err != nil {
+ panic(err)
+ }
+ return
+
And it seems that your example works.
@ivokub thanks! It works now!! It only took 14s to prove, with 4,772,531 constraints (the Circom circuit has around 35k and 42 public signals) on my laptop (AMD Ryzen 7 7840U + 64 GiB). That's very outstanding.
I'll package it correctly and provide here the link for the final repository, so other ppl can take advantage of it.
That would be great. I'll see if I could incorporate as an example also in gnark, it has been asked several times.
Meanwhile, I'm also starting to prepare an example how to reduce the number of public inputs, so instead of providing all 42 public inputs you could only provide a single public input H(input1, ..., input42)
and then everything else as a private input. Then in circuit you could check that the hashed input corresponds to the input1,...
. I think currently the circuit size is dominated by a large MSM and this way it would boil down to a single scalarmul. It could potentially reduce the outer circuit size considerably, making the proving time even faster.
And this is in the end really useful for Solidity verifier also to reduce the calldata.
Ok, I've cleaned up and refactored the code. Here is the final repository. https://github.com/vocdoni/circom2gnark There are still some functions from the parser that can be improved, but nothing important as far as I understand.
@ivokub feel free to send a PR with improvements or make any comment you like.
That would be great. I'll see if I could incorporate as an example also in gnark, it has been asked several times.
Sure, let me know if I can help.
Meanwhile, I'm also starting to prepare an example how to reduce the number of public inputs, so instead of providing all 42 public inputs you could only provide a single public input H(input1, ..., input42) and then everything else as a private input. Then in circuit you could check that the hashed input corresponds to the input1,.... I think currently the circuit size is dominated by a large MSM and this way it would boil down to a single scalarmul. It could potentially reduce the outer circuit size considerably, making the proving time even faster.
That makes sense, I've seen this technique used many times. Please notify me if you implement the example, so I incorporate it.
Ok, I've cleaned up and refactored the code. Here is the final repository. https://github.com/vocdoni/circom2gnark There are still some functions from the parser that can be improved, but nothing important as far as I understand.
@ivokub feel free to send a PR with improvements or make any comment you like.
That would be great. I'll see if I could incorporate as an example also in gnark, it has been asked several times.
Sure, let me know if I can help.
Meanwhile, I'm also starting to prepare an example how to reduce the number of public inputs, so instead of providing all 42 public inputs you could only provide a single public input H(input1, ..., input42) and then everything else as a private input. Then in circuit you could check that the hashed input corresponds to the input1,.... I think currently the circuit size is dominated by a large MSM and this way it would boil down to a single scalarmul. It could potentially reduce the outer circuit size considerably, making the proving time even faster.
That makes sense, I've seen this technique used many times. Please notify me if you implement the example, so I incorporate it.
Thanks. I'll be traveling, but will have a look soon.
Meanwhile, I added an example on how to use the hash of the public inputs to reduce the recursive verification cost. See https://github.com/Consensys/gnark/pull/1311. I'll also think about if I should add another example how it would look in a recursive circuit.
I am not sure, but I might have located a little issue on the Placeolder helpers.
While doing another example, this time using native recursion (bls12-377 + bw6-761) to create a second SNARK that aggregate several proofs previously recursively transformed from bn254 (circom) to bls12-377 Gnark.
When trying to generate the proof on the bw6-716 curve to aggregate 10 bls12-377 proofs, I receive the error:
2024/11/06 14:03:51 Failed to test solve circuit: define: assert proof: invalid number of commitments, got 1, expected 0
So I have printed the number of commitment keys and got the following:
Number of commitments Vk: 1
Number of commitments Vk placeholder: 0
Number of commitments Proof: 1
Number of commitments Proof placeholder: 1
The placeholder Vk is created using stdgroth16.PlaceholderVerifyingKey[sw_bls12377.G1Affine, sw_bls12377.G2Affine, sw_bls12377.GT](ccs)
I tried to manually modify the placeholder Vk with: placeholderVk.G1.K = make([]sw_bls12377.G1Affine, len(placeholderVk.G1.K))
. Then it becomes 1, but got blocked on the second check:which compares PublicAndCommitmentCommited: invalid number of commitment keys, got 1, expected 0
I can keep trying to modifying manually the Vk, but it does not feel good...
I can perfectly verify all bls12-377 independent proofs with the standard groth16.Verify() function.
Am I doing something wrong or there is something weird happening with the bw6 recursion?
I'm very stuck here. I would really appreciate some help (@ivokub)
What I'm trying to achieve is a native recursion, so aggregating circom proofs should be faster.
bn254/circom ---emulated---> bls12377/gnark ----native---> bw6_761/gnark
The new example code I'm building can be found here: https://github.com/vocdoni/circom2gnark/tree/main/example_native_aggregation
I'm trying to do the things exactly as the native example, but for some reason the proof generation fails.
I'm looking into
I'm very stuck here. I would really appreciate some help (@ivokub)
What I'm trying to achieve is a native recursion, so aggregating circom proofs should be faster.
bn254/circom ---emulated---> bls12377/gnark ----native---> bw6_761/gnark
The new example code I'm building can be found here: https://github.com/vocdoni/circom2gnark/tree/main/example_native_aggregation
I'm trying to do the things exactly as the native example, but for some reason the proof generation fails.
Sorry for the late reply - I have been at Devcon.
It took me some time to debug, but I found it. I think our documentation is not super clean, we wrote it before we had the support for commitments in Groth16 recursion. So, the issue is that when we have a commitment, then we need to hash it and instead of the default hash function when we don't use recursion, we use in circuit different hash function which is snark friendly. But we need to define it at proving time.
So, with the following modification:
diff --git a/example_native_aggregation/circomproofs.go b/example_native_aggregation/circomproofs.go
index d4a25c1..3d274c8 100644
--- a/example_native_aggregation/circomproofs.go
+++ b/example_native_aggregation/circomproofs.go
@@ -11,6 +11,7 @@ import (
"github.com/consensys/gnark/backend/witness"
"github.com/consensys/gnark/constraint"
"github.com/consensys/gnark/frontend"
+ stdgroth16 "github.com/consensys/gnark/std/recursion/groth16"
"github.com/consensys/gnark/test"
"github.com/vocdoni/circom2gnark/parser"
)
@@ -122,7 +123,7 @@ func circom2gnarkRecursiveBls12377(proofData, vkData, publicSignalsData []byte,
// Create the proof
fmt.Println("Generating a recursive proof BLS12-377 of an independent Circom proof...")
startTime := time.Now()
- proof, err := groth16.Prove(ccs, pk, witnessFull)
+ proof, err := groth16.Prove(ccs, pk, witnessFull, stdgroth16.GetNativeProverOptions(ecc.BW6_761.ScalarField(), ecc.BLS12_377.ScalarField()))
if err != nil {
log.Fatalf("Failed to create proof: %v", err)
}
@@ -131,7 +132,7 @@ func circom2gnarkRecursiveBls12377(proofData, vkData, publicSignalsData []byte,
// Verify the proof
fmt.Println("Verifying...")
startTime = time.Now()
- err = groth16.Verify(proof, vk, publicWitness)
+ err = groth16.Verify(proof, vk, publicWitness, stdgroth16.GetNativeVerifierOptions(ecc.BW6_761.ScalarField(), ecc.BLS12_377.ScalarField()))
if err != nil {
log.Fatalf("Failed to verify proof: %v", err)
}
I tried it with a smaller inner circuit, but I think you should be able to reproduce for the big circuit also. Let me know if it works or not, and I'll have another look.
Another thing what I would recommend - for making proving key reading faster you can use the gnarkio.BinaryDumper
interfaces which are faster. See https://pkg.go.dev/github.com/consensys/gnark@v0.11.0/backend/groth16#ProvingKey
I'm looking into
I'm very stuck here. I would really appreciate some help (@ivokub) What I'm trying to achieve is a native recursion, so aggregating circom proofs should be faster.
bn254/circom ---emulated---> bls12377/gnark ----native---> bw6_761/gnark
The new example code I'm building can be found here: https://github.com/vocdoni/circom2gnark/tree/main/example_native_aggregation I'm trying to do the things exactly as the native example, but for some reason the proof generation fails.Sorry for the late reply - I have been at Devcon.
It took me some time to debug, but I found it. I think our documentation is not super clean, we wrote it before we had the support for commitments in Groth16 recursion. So, the issue is that when we have a commitment, then we need to hash it and instead of the default hash function when we don't use recursion, we use in circuit different hash function which is snark friendly. But we need to define it at proving time.
So, with the following modification:
diff --git a/example_native_aggregation/circomproofs.go b/example_native_aggregation/circomproofs.go index d4a25c1..3d274c8 100644 --- a/example_native_aggregation/circomproofs.go +++ b/example_native_aggregation/circomproofs.go @@ -11,6 +11,7 @@ import ( "github.com/consensys/gnark/backend/witness" "github.com/consensys/gnark/constraint" "github.com/consensys/gnark/frontend" + stdgroth16 "github.com/consensys/gnark/std/recursion/groth16" "github.com/consensys/gnark/test" "github.com/vocdoni/circom2gnark/parser" ) @@ -122,7 +123,7 @@ func circom2gnarkRecursiveBls12377(proofData, vkData, publicSignalsData []byte, // Create the proof fmt.Println("Generating a recursive proof BLS12-377 of an independent Circom proof...") startTime := time.Now() - proof, err := groth16.Prove(ccs, pk, witnessFull) + proof, err := groth16.Prove(ccs, pk, witnessFull, stdgroth16.GetNativeProverOptions(ecc.BW6_761.ScalarField(), ecc.BLS12_377.ScalarField())) if err != nil { log.Fatalf("Failed to create proof: %v", err) } @@ -131,7 +132,7 @@ func circom2gnarkRecursiveBls12377(proofData, vkData, publicSignalsData []byte, // Verify the proof fmt.Println("Verifying...") startTime = time.Now() - err = groth16.Verify(proof, vk, publicWitness) + err = groth16.Verify(proof, vk, publicWitness, stdgroth16.GetNativeVerifierOptions(ecc.BW6_761.ScalarField(), ecc.BLS12_377.ScalarField())) if err != nil { log.Fatalf("Failed to verify proof: %v", err) }
I tried it with a smaller inner circuit, but I think you should be able to reproduce for the big circuit also. Let me know if it works or not, and I'll have another look.
Another thing what I would recommend - for making proving key reading faster you can use the
gnarkio.BinaryDumper
interfaces which are faster. See https://pkg.go.dev/github.com/consensys/gnark@v0.11.0/backend/groth16#ProvingKey
Ok, it works now! Amazing @ivokub I can aggregate 10 circom proofs (already transformed to bls12-377) in 0.4 seconds (400k constraints). This is pretty good.
Thanks for your help! I hope this digging on recursion can be useful to improve gnark documentation.
Hey @ivokub hope you are enjoying devcon, good luck with your talk tomorrow. There is no rush to reply, take your time.
I'm implementing the last part of my example, which is going back from bw6-761 to bn254, so the final aggregation of Circom proofs is Ethereum friendly and can be verified onchain.
To this end I've impemented a new circuit:
// AggregateProofCircuitBN254 is the circuit that verifies the proof aggregation using BN254 curve
type AggregateProofCircuitBN254 struct {
Proof stdgroth16.Proof[sw_bw6761.G1Affine, sw_bw6761.G2Affine]
VerifyingKey stdgroth16.VerifyingKey[sw_bw6761.G1Affine, sw_bw6761.G2Affine, sw_bw6761.GTEl]
PublicInputs stdgroth16.Witness[sw_bw6761.ScalarField] `gnark:",public"`
}
func (c *AggregateProofCircuitBN254) Define(api frontend.API) error {
verifier, err := stdgroth16.NewVerifier[sw_bw6761.ScalarField, sw_bw6761.G1Affine, sw_bw6761.G2Affine, sw_bw6761.GTEl](api)
if err != nil {
return fmt.Errorf("new verifier: %w", err)
}
return verifier.AssertProof(c.VerifyingKey, c.Proof, c.PublicInputs)
}
And so, I'm trying to generate a proof of emulated bw6761 within bn254. But no luck again.
23:42:35 ERR error="no modular inverse" nbConstraints=10859962
And the test also panics.
For testing, I tried to switch the native recursion curves to BLS24-315 and BW6-633 instead, but there is no emulated algebra for bw6-633 for some reason.
I've pushed the new code https://github.com/vocdoni/circom2gnark/blob/main/example_native_aggregation/aggregatebn254.go
Again, any help is welcome. Thank you.
Forget it, works! Again, stdgroth16.WithCompleteArithmetic() was missing.
However, making this last step, heavily slows down the process. There are 96 public inputs, so I expect that using the Hash(publicInputs) will reduce the number of constraints. But still, do you see a way to optimize the process a little more?
Hello. I managed to create a Parser for Circom/SnarkJS proof, verifying key and inputs to Gnark. I do believe this is a nice to have feature because there is nothing like Circom for browsers (in terms of cmpatibility, memory consumption and so on). The idea behind this is to be able to aggregate Circom proofs using Gnark and the bn254 recursion.
The current status is that the Parser works and I can verify the Circom Proof using the groth16 Verifier from Gnark. That's great!
However, I'm struggling trying to build a second SNARK to prove recursively the Circom Proof (I have been following the example on std/recursion/groth16). So at this point I'm blocked and so asking for help.
Everything seems to work fine until the proof is generated, then after some time computing, it fails:
12:30:22 ERR error="no modular inverse" nbConstraints=3510919
.Since the non-recursive verification of the proof works, I do believe the issue is when formatting the types to the new recursion types. I have tried several ways to do it, but the result is always the same.
Any hint that helps me resolve this issue would be very welcome.
The code is messy and dirty, I'll package it correctly if I manage to make it work :)
https://github.com/p4u/circomgnark
(this log output is from the branch
with_circuit_definition
)