search

lambdaclass / cairo-vm

cairo-vm is a Rust implementation of the Cairo VM. Cairo (CPU Algebraic Intermediate Representation) is a programming language for writing provable programs, where one party can prove to another that a certain computation was executed correctly without the need for this party to re-execute the same program.
https://lambdaclass.github.io/cairo-vm
Apache License 2.0
514 stars 144 forks source link

Test adding programs using non squashed Felt252Dict #1797

Closed FrancoGiachetta closed 2 months ago

FrancoGiachetta commented 3 months ago

Add non squashed Felt252Dict tests

Description

PR #1721 modified felt tests so that the work with SquashedFelt252Dict. This PR adds the same tests with the difference that they use Felt252Dict.

Checklist

github-actions[bot] commented 3 months ago 
**Hyper Thereading Benchmark results**

hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     25.436 s ±  0.090 s    [User: 24.661 s, System: 0.774 s]
  Range (min … max):   25.373 s … 25.500 s    2 runs

Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     25.495 s ±  0.044 s    [User: 24.684 s, System: 0.810 s]
  Range (min … max):   25.463 s … 25.526 s    2 runs

Summary
  'hyper_threading_main threads: 1' ran
    1.00 ± 0.00 times faster than 'hyper_threading_pr threads: 1'

hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     14.068 s ±  0.014 s    [User: 25.108 s, System: 0.781 s]
  Range (min … max):   14.058 s … 14.078 s    2 runs

Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     14.109 s ±  0.050 s    [User: 25.154 s, System: 0.794 s]
  Range (min … max):   14.074 s … 14.145 s    2 runs

Summary
  'hyper_threading_main threads: 2' ran
    1.00 ± 0.00 times faster than 'hyper_threading_pr threads: 2'

hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):     10.658 s ±  0.374 s    [User: 37.397 s, System: 0.980 s]
  Range (min … max):   10.394 s … 10.923 s    2 runs

Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):     10.541 s ±  0.668 s    [User: 37.878 s, System: 0.983 s]
  Range (min … max):   10.069 s … 11.014 s    2 runs

Summary
  'hyper_threading_pr threads: 4' ran
    1.01 ± 0.07 times faster than 'hyper_threading_main threads: 4'

hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):     10.287 s ±  0.042 s    [User: 38.195 s, System: 0.973 s]
  Range (min … max):   10.257 s … 10.317 s    2 runs

Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):     10.472 s ±  0.284 s    [User: 37.770 s, System: 0.927 s]
  Range (min … max):   10.271 s … 10.674 s    2 runs

Summary
  'hyper_threading_main threads: 6' ran
    1.02 ± 0.03 times faster than 'hyper_threading_pr threads: 6'

hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):     10.269 s ±  0.028 s    [User: 38.060 s, System: 0.984 s]
  Range (min … max):   10.249 s … 10.289 s    2 runs

Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):     10.331 s ±  0.137 s    [User: 37.899 s, System: 1.016 s]
  Range (min … max):   10.234 s … 10.427 s    2 runs

Summary
  'hyper_threading_main threads: 8' ran
    1.01 ± 0.01 times faster than 'hyper_threading_pr threads: 8'

hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):     10.303 s ±  0.300 s    [User: 38.422 s, System: 1.044 s]
  Range (min … max):   10.091 s … 10.515 s    2 runs

Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):     10.237 s ±  0.102 s    [User: 38.505 s, System: 1.073 s]
  Range (min … max):   10.164 s … 10.309 s    2 runs

Summary
  'hyper_threading_pr threads: 16' ran
    1.01 ± 0.03 times faster than 'hyper_threading_main threads: 16'
codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.89%. Comparing base (309c8a7) to head (9f111a6).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1797 +/- ## ========================================== + Coverage 94.79% 94.89% +0.09% ========================================== Files 102 102 Lines 40150 40151 +1 ========================================== + Hits 38059 38100 +41 + Misses 2091 2051 -40 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 3 months ago

Benchmark Results for unmodified programs :rocket:

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 2.006 ± 0.053 1.980 2.154 1.02 ± 0.03
head big_factorial 1.974 ± 0.009 1.961 1.986 1.00
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 1.918 ± 0.022 1.896 1.964 1.00
head big_fibonacci 1.939 ± 0.015 1.911 1.959 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 7.286 ± 0.142 7.059 7.449 1.00
head blake2s_integration_benchmark 7.628 ± 0.872 7.060 9.376 1.05 ± 0.12
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 2.024 ± 0.030 1.964 2.066 1.00
head compare_arrays_200000 2.075 ± 0.061 2.031 2.229 1.03 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 1.340 ± 0.017 1.319 1.368 1.00
head dict_integration_benchmark 1.346 ± 0.017 1.320 1.379 1.00 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.175 ± 0.013 1.151 1.190 1.00 ± 0.02
head field_arithmetic_get_square_benchmark 1.175 ± 0.012 1.158 1.199 1.00
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 7.309 ± 0.132 7.139 7.521 1.00
head integration_builtins 7.356 ± 0.189 7.171 7.812 1.01 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 7.540 ± 0.107 7.373 7.651 1.00
head keccak_integration_benchmark 7.777 ± 0.506 7.355 9.186 1.03 ± 0.07
Command Mean [s] Min [s] Max [s] Relative
base linear_search 2.049 ± 0.030 2.001 2.084 1.00
head linear_search 2.069 ± 0.040 2.030 2.143 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 1.425 ± 0.010 1.414 1.447 1.00
head math_cmp_and_pow_integration_benchmark 1.444 ± 0.042 1.422 1.561 1.01 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 1.410 ± 0.063 1.361 1.584 1.02 ± 0.05
head math_integration_benchmark 1.382 ± 0.013 1.364 1.409 1.00
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 1.143 ± 0.005 1.137 1.152 1.00
head memory_integration_benchmark 1.168 ± 0.007 1.156 1.180 1.02 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 1.480 ± 0.016 1.454 1.503 1.00
head operations_with_data_structures_benchmarks 1.498 ± 0.032 1.458 1.576 1.01 ± 0.02
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 512.3 ± 2.8 507.0 517.1 1.00
head pedersen 513.0 ± 7.4 497.2 527.4 1.00 ± 0.02
Command Mean [ms] Min [ms] Max [ms] Relative
base poseidon_integration_benchmark 724.3 ± 13.6 694.1 739.3 1.01 ± 0.02
head poseidon_integration_benchmark 716.2 ± 8.7 702.5 729.7 1.00
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 1.723 ± 0.022 1.694 1.768 1.00
head secp_integration_benchmark 1.727 ± 0.012 1.710 1.755 1.00 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base set_integration_benchmark 621.4 ± 10.3 601.0 636.7 1.00
head set_integration_benchmark 622.7 ± 7.5 614.7 635.6 1.00 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 4.015 ± 0.065 3.932 4.174 1.00 ± 0.02
head uint256_integration_benchmark 4.006 ± 0.059 3.899 4.077 1.00
FrancoGiachetta commented 3 months ago

@pefontana changes were done. Sorry about the files, seems I've had some trouble renaming the files.