snarkify / sirius

A Plonkish folding framework for Incrementally Verifiable Computation (IVC).
MIT License
106 stars 15 forks source link

bench(poseidon): benchmark the ivc folding time when poseidon step circuit size changes #249

Closed chaosma closed 1 month ago

chaosma commented 1 month ago

Suppose poseidon hash function is given by y=H(x). As for the benchmark, we can test the following.

Given a step circuit size n, i.e. each step of the circuit will do poseidon hash n times. In this case, the step circuit is of the form y=H^n(x)=H(H(...(H(x)).

We want to see the how the step circuit size n will affect the IVC folding time. For example, we want to know when the step circuit size (the row number) is similar to ivc circuit size (the row number) and when the step circuit size is larger than ivc folding circuit size, how it will affect the folding time.

cyphersnake commented 1 month ago

image

Input here is repeat_count

cyphersnake commented 1 month ago

WIP: The poseidon-hash occupies a different number of rows each time, so it is possible to test the case of occupying more rows by StepCircuit.

cyphersnake commented 1 month ago

With k=20 & step=5 criterion.tar.gz

cyphersnake commented 1 month ago

violin sfc is reached somewhere around 19,999.

The Y axis is the experiment, the first digit is the number of poseidon-hash repetitions, the second digit is log2(filled_rows_count) X-axis is the execution time

In general, there should be points with average time, but I seem to have made too small a step and critertion got bad, it can't collect the final graph for several hours. But the nature of the dependence can still be seen even here.

cyphersnake commented 1 month ago

violin

Is any more data needed or can the issue be closed?

chaosma commented 1 month ago

It looks good to me, which is expected behavior.

drouyang commented 1 month ago

could anyone create a notion or hackmd writeup summarizing performance data and our observations?

cyphersnake commented 1 month ago

could anyone create a notion or hackmd writeup summarizing performance data and our observations?

Well this task is specifically a request from Chao for certain information, which you can see above. If we want a summary, we need to define what will be included in it.

I wanted to do just such an overview, with definition of key parameters, but with re-focus on Protogalaxy+CycleFold we postponed this task as irrelevant.

drouyang commented 1 month ago

I mean a write-up to summarize Sirius progress and performances as a milestone and help everyone understand where Sirius is at and what are the properties. Like a short paper.

When you implement something, conduct experiments, you create a simple write-up. That's a common practice when doing R&D. Issue and PRs are for engineering, write-ups are for reporting and communication (both internally and externally)