search

lambdaclass / cairo-vm

cairo-vm is a Rust implementation of the Cairo VM. Cairo (CPU Algebraic Intermediate Representation) is a programming language for writing provable programs, where one party can prove to another that a certain computation was executed correctly without the need for this party to re-execute the same program.
https://lambdaclass.github.io/cairo-vm
Apache License 2.0
485 stars 133 forks source link

Bump `starknet-types-core` version + use the lib's poseidon hash #1734

Closed fmoletta closed 1 month ago

fmoletta commented 2 months ago
github-actions[bot] commented 2 months ago 
**Hyper Thereading Benchmark results**

hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     27.831 s ±  0.031 s    [User: 27.049 s, System: 0.780 s]
  Range (min … max):   27.809 s … 27.853 s    2 runs

Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     27.067 s ±  0.027 s    [User: 26.259 s, System: 0.806 s]
  Range (min … max):   27.048 s … 27.086 s    2 runs

Summary
  'hyper_threading_pr threads: 1' ran
    1.03 ± 0.00 times faster than 'hyper_threading_main threads: 1'

hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     15.075 s ±  0.049 s    [User: 27.723 s, System: 0.814 s]
  Range (min … max):   15.041 s … 15.110 s    2 runs

Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     15.099 s ±  0.021 s    [User: 26.910 s, System: 0.794 s]
  Range (min … max):   15.084 s … 15.114 s    2 runs

Summary
  'hyper_threading_main threads: 2' ran
    1.00 ± 0.00 times faster than 'hyper_threading_pr threads: 2'

hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):     10.813 s ±  0.224 s    [User: 38.442 s, System: 0.989 s]
  Range (min … max):   10.655 s … 10.971 s    2 runs

Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):     11.331 s ±  0.084 s    [User: 38.243 s, System: 0.958 s]
  Range (min … max):   11.272 s … 11.391 s    2 runs

Summary
  'hyper_threading_main threads: 4' ran
    1.05 ± 0.02 times faster than 'hyper_threading_pr threads: 4'

hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):     10.607 s ±  0.275 s    [User: 39.121 s, System: 1.012 s]
  Range (min … max):   10.412 s … 10.801 s    2 runs

Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):     10.896 s ±  0.290 s    [User: 38.231 s, System: 1.015 s]
  Range (min … max):   10.691 s … 11.101 s    2 runs

Summary
  'hyper_threading_main threads: 6' ran
    1.03 ± 0.04 times faster than 'hyper_threading_pr threads: 6'

hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):     10.600 s ±  0.137 s    [User: 39.161 s, System: 1.048 s]
  Range (min … max):   10.503 s … 10.697 s    2 runs

Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):     10.489 s ±  0.288 s    [User: 38.687 s, System: 0.979 s]
  Range (min … max):   10.286 s … 10.693 s    2 runs

Summary
  'hyper_threading_pr threads: 8' ran
    1.01 ± 0.03 times faster than 'hyper_threading_main threads: 8'

hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):     10.574 s ±  0.197 s    [User: 39.549 s, System: 1.039 s]
  Range (min … max):   10.435 s … 10.713 s    2 runs

Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):     10.409 s ±  0.272 s    [User: 39.024 s, System: 1.058 s]
  Range (min … max):   10.217 s … 10.601 s    2 runs

Summary
  'hyper_threading_pr threads: 16' ran
    1.02 ± 0.03 times faster than 'hyper_threading_main threads: 16'
codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.78%. Comparing base (05352b1) to head (1e47295). Report is 1 commits behind head on main.

:exclamation: Current head 1e47295 differs from pull request most recent head a65b44a

Please upload reports for the commit a65b44a to get more accurate results.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1734 +/- ## ======================================== Coverage 94.78% 94.78% ======================================== Files 101 101 Lines 39005 38728 -277 ======================================== - Hits 36969 36709 -260 + Misses 2036 2019 -17 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 2 months ago

Benchmark Results for unmodified programs :rocket:

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 2.102 ± 0.020 2.083 2.141 1.00
head big_factorial 2.110 ± 0.019 2.092 2.144 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 2.068 ± 0.028 2.027 2.124 1.00 ± 0.02
head big_fibonacci 2.067 ± 0.030 2.036 2.140 1.00
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 8.098 ± 0.248 7.769 8.668 1.03 ± 0.04
head blake2s_integration_benchmark 7.858 ± 0.125 7.675 8.041 1.00
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 2.205 ± 0.031 2.172 2.248 1.00 ± 0.02
head compare_arrays_200000 2.204 ± 0.033 2.164 2.256 1.00
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 1.443 ± 0.010 1.428 1.459 1.00
head dict_integration_benchmark 1.453 ± 0.008 1.438 1.464 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.335 ± 0.027 1.302 1.380 1.07 ± 0.03
head field_arithmetic_get_square_benchmark 1.245 ± 0.018 1.224 1.279 1.00
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 7.888 ± 0.180 7.705 8.232 1.00 ± 0.03
head integration_builtins 7.883 ± 0.101 7.733 8.018 1.00
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 8.345 ± 0.145 8.075 8.464 1.03 ± 0.03
head keccak_integration_benchmark 8.123 ± 0.204 7.898 8.498 1.00
Command Mean [s] Min [s] Max [s] Relative
base linear_search 2.173 ± 0.024 2.121 2.214 1.01 ± 0.02
head linear_search 2.147 ± 0.021 2.120 2.194 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 1.731 ± 0.009 1.713 1.743 1.17 ± 0.02
head math_cmp_and_pow_integration_benchmark 1.482 ± 0.021 1.466 1.538 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 1.633 ± 0.011 1.621 1.655 1.11 ± 0.01
head math_integration_benchmark 1.478 ± 0.013 1.463 1.509 1.00
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 1.225 ± 0.007 1.215 1.234 1.00
head memory_integration_benchmark 1.229 ± 0.008 1.219 1.246 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 1.873 ± 0.055 1.835 2.023 1.19 ± 0.04
head operations_with_data_structures_benchmarks 1.572 ± 0.022 1.552 1.628 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 529.8 ± 6.1 522.4 541.3 1.00
head pedersen 530.6 ± 7.8 522.7 544.2 1.00 ± 0.02
Command Mean [ms] Min [ms] Max [ms] Relative
base poseidon_integration_benchmark 984.6 ± 3.4 981.8 993.4 1.29 ± 0.01
head poseidon_integration_benchmark 763.0 ± 4.7 756.8 772.4 1.00
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 1.901 ± 0.014 1.878 1.918 1.03 ± 0.01
head secp_integration_benchmark 1.851 ± 0.017 1.835 1.879 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base set_integration_benchmark 654.7 ± 4.4 647.2 664.4 1.00
head set_integration_benchmark 675.6 ± 3.6 669.2 680.6 1.03 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 4.416 ± 0.092 4.262 4.567 1.00
head uint256_integration_benchmark 4.473 ± 0.232 4.319 5.021 1.01 ± 0.06
Oppen commented 2 months ago
image

Not good.

pefontana commented 2 months ago
image

Not good.

Yes, we should wait the types-rs update https://github.com/starknet-io/types-rs/pull/64

And see the results