search

TritonVM / triton-vm

Triton is a virtual machine that comes with Algebraic Execution Tables (AET) and Arithmetic Intermediate Representations (AIR) for use in combination with a STARK proof system.
https://triton-vm.org
Apache License 2.0
243 stars 36 forks source link

Add Statistics to Triton Profiler #107

Open aszepieniec opened 1 year ago

aszepieniec commented 1 year ago

Currently, TritonProfiler takes timings from one execution. However, for robust benchmarking it is interesting to see statistics like mean an standard deviation across a larger number of samples.

aszepieniec commented 1 year ago

@jan-ferdinand suggested a related feature. Currently, the profile from the zeroth iteration of a loop is stored and the profiles of later iterations are not collected. It could be possible to merge these profiles and collect statistics over them.

A lot of the machinery for merging task profiles is common across both tasks. It would be a shame to repeat code.

jan-ferdinand commented 1 year ago

Similar but different: it would be nice to declare the category of a certain task. That way, the measurements for LDE 1, 2, and 3, or for building Merkle trees, can be summed by the profiler.

edit: Categories can be tracked as per 7edd1a2c9509367e6cd19e8811093862e3b04f0d.

jan-ferdinand commented 4 months ago

As per #281, profiling task's runtimes are accumulated across loops. This puts in place most of the bookkeeping required for computing statistics. The last tasks to close this issue are: