search

filecoin-project / bellperson

zk-SNARK library
Other
190 stars 120 forks source link

Large batch sizes with SupraSeal #320

Open vmx opened 11 months ago

vmx commented 11 months ago

When using proof batch sizes > 10, things will take a lot of memory. There's now a memory optimized code path which is only enabled for SupraSeal as it takes advantage of the shape of the Filecoin circuits. It may not be as efficient with other circuits.

This PR doesn't change the current PoRep behaviour, it is a different code path only if the proofs size is >10, which is the case for e.g. non-interactive PoRep. With these changes, non-interactive PoRep, which has about 12.6x times the proofs to do compared to the interactive PoRep, runs only about 8x slower, while not taking more memory.

vmx commented 11 months ago

Sorry I accidentally pushed a change, but I reverted back to what it was.

vmx commented 11 months ago

I've converted it to draft, as it needs more work. The CPU usage is not ideal, sometimes the Rayon thread pool is used by the synthesis, which then blocks the representation conversion. Ideally the Synthesis runs whenever the GPU is used.