search

lambdaclass / cairo-vm

cairo-vm is a Rust implementation of the Cairo VM. Cairo (CPU Algebraic Intermediate Representation) is a programming language for writing provable programs, where one party can prove to another that a certain computation was executed correctly without the need for this party to re-execute the same program.
https://lambdaclass.github.io/cairo-vm
Apache License 2.0
514 stars 144 forks source link

refactor: Move the VM back to the CairoRunner #1743

Closed fmoletta closed 4 months ago

fmoletta commented 5 months ago

This change was made quite a while ago because of cairo-vm-py. As support for that crate has been dropped, we can revert this change and have a cleaner, simpler and less confusing api.

github-actions[bot] commented 5 months ago 
**Hyper Thereading Benchmark results**

hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     27.068 s ±  0.072 s    [User: 26.242 s, System: 0.824 s]
  Range (min … max):   27.017 s … 27.119 s    2 runs

Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     27.500 s ±  0.060 s    [User: 26.747 s, System: 0.753 s]
  Range (min … max):   27.458 s … 27.543 s    2 runs

Summary
  'hyper_threading_main threads: 1' ran
    1.02 ± 0.00 times faster than 'hyper_threading_pr threads: 1'

hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     14.597 s ±  0.063 s    [User: 26.866 s, System: 0.751 s]
  Range (min … max):   14.553 s … 14.642 s    2 runs

Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     14.846 s ±  0.018 s    [User: 27.037 s, System: 0.819 s]
  Range (min … max):   14.833 s … 14.859 s    2 runs

Summary
  'hyper_threading_main threads: 2' ran
    1.02 ± 0.00 times faster than 'hyper_threading_pr threads: 2'

hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):     11.135 s ±  0.221 s    [User: 38.775 s, System: 0.970 s]
  Range (min … max):   10.979 s … 11.292 s    2 runs

Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):     10.971 s ±  0.287 s    [User: 39.021 s, System: 1.018 s]
  Range (min … max):   10.768 s … 11.174 s    2 runs

Summary
  'hyper_threading_pr threads: 4' ran
    1.01 ± 0.03 times faster than 'hyper_threading_main threads: 4'

hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):     10.689 s ±  0.242 s    [User: 39.046 s, System: 1.018 s]
  Range (min … max):   10.518 s … 10.860 s    2 runs

Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):     10.832 s ±  0.154 s    [User: 39.347 s, System: 1.036 s]
  Range (min … max):   10.724 s … 10.941 s    2 runs

Summary
  'hyper_threading_main threads: 6' ran
    1.01 ± 0.03 times faster than 'hyper_threading_pr threads: 6'

hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):     10.485 s ±  0.102 s    [User: 39.364 s, System: 1.026 s]
  Range (min … max):   10.414 s … 10.557 s    2 runs

Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):     10.575 s ±  0.018 s    [User: 39.585 s, System: 1.002 s]
  Range (min … max):   10.562 s … 10.587 s    2 runs

Summary
  'hyper_threading_main threads: 8' ran
    1.01 ± 0.01 times faster than 'hyper_threading_pr threads: 8'

hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):     10.625 s ±  0.102 s    [User: 39.485 s, System: 1.040 s]
  Range (min … max):   10.553 s … 10.697 s    2 runs

Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):     10.613 s ±  0.105 s    [User: 39.940 s, System: 1.076 s]
  Range (min … max):   10.538 s … 10.687 s    2 runs

Summary
  'hyper_threading_pr threads: 16' ran
    1.00 ± 0.01 times faster than 'hyper_threading_main threads: 16'
codecov[bot] commented 5 months ago

Codecov Report

Attention: Patch coverage is 94.75806% with 13 lines in your changes are missing coverage. Please review.

Project coverage is 94.79%. Comparing base (b25dae7) to head (013eed5).

Files Patch % Lines
vm/src/cairo_run.rs 82.66% 13 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1743 +/- ## ========================================== + Coverage 94.76% 94.79% +0.02% ========================================== Files 101 101 Lines 38826 38723 -103 ========================================== - Hits 36795 36709 -86 + Misses 2031 2014 -17 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 5 months ago

Benchmark Results for unmodified programs :rocket:

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 2.073 ± 0.024 2.051 2.132 1.00
head big_factorial 2.081 ± 0.008 2.070 2.095 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 2.020 ± 0.056 1.982 2.174 1.00 ± 0.03
head big_fibonacci 2.020 ± 0.020 1.998 2.069 1.00
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 7.642 ± 0.077 7.519 7.737 1.00
head blake2s_integration_benchmark 7.673 ± 0.344 7.396 8.597 1.00 ± 0.05
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 2.153 ± 0.034 2.119 2.208 1.00
head compare_arrays_200000 2.154 ± 0.041 2.129 2.268 1.00 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 1.428 ± 0.028 1.412 1.506 1.00
head dict_integration_benchmark 1.432 ± 0.036 1.406 1.521 1.00 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.287 ± 0.006 1.279 1.295 1.00
head field_arithmetic_get_square_benchmark 1.293 ± 0.007 1.282 1.302 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 7.658 ± 0.063 7.551 7.737 1.00
head integration_builtins 7.666 ± 0.237 7.474 8.262 1.00 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 7.879 ± 0.074 7.779 7.981 1.00
head keccak_integration_benchmark 7.918 ± 0.239 7.676 8.517 1.00 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base linear_search 2.082 ± 0.019 2.062 2.119 1.00
head linear_search 2.106 ± 0.010 2.094 2.123 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 1.706 ± 0.011 1.695 1.733 1.00
head math_cmp_and_pow_integration_benchmark 1.711 ± 0.004 1.706 1.719 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 1.605 ± 0.007 1.596 1.618 1.00
head math_integration_benchmark 1.611 ± 0.003 1.606 1.616 1.00 ± 0.00
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 1.204 ± 0.005 1.196 1.211 1.00
head memory_integration_benchmark 1.216 ± 0.008 1.209 1.232 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 1.831 ± 0.016 1.806 1.868 1.00
head operations_with_data_structures_benchmarks 1.843 ± 0.026 1.819 1.903 1.01 ± 0.02
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 517.9 ± 2.2 515.1 522.0 1.00
head pedersen 524.4 ± 10.1 517.1 549.4 1.01 ± 0.02
Command Mean [ms] Min [ms] Max [ms] Relative
base poseidon_integration_benchmark 967.3 ± 6.8 959.4 979.1 1.00
head poseidon_integration_benchmark 975.7 ± 23.1 964.1 1040.3 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 1.873 ± 0.030 1.848 1.933 1.01 ± 0.02
head secp_integration_benchmark 1.856 ± 0.008 1.843 1.868 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base set_integration_benchmark 646.2 ± 10.1 639.7 674.3 1.00
head set_integration_benchmark 667.4 ± 1.3 665.4 669.3 1.03 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 4.245 ± 0.052 4.151 4.329 1.00
head uint256_integration_benchmark 4.248 ± 0.068 4.139 4.371 1.00 ± 0.02