Closed fmoletta closed 5 months ago
**Hyper Thereading Benchmark results**
hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
Time (mean ± σ): 26.086 s ± 0.127 s [User: 25.410 s, System: 0.673 s]
Range (min … max): 25.996 s … 26.176 s 2 runs
Benchmark 2: hyper_threading_pr threads: 1
Time (mean ± σ): 26.035 s ± 0.056 s [User: 25.355 s, System: 0.677 s]
Range (min … max): 25.996 s … 26.074 s 2 runs
Summary
'hyper_threading_pr threads: 1' ran
1.00 ± 0.01 times faster than 'hyper_threading_main threads: 1'
hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
Time (mean ± σ): 13.799 s ± 0.058 s [User: 25.665 s, System: 0.720 s]
Range (min … max): 13.757 s … 13.840 s 2 runs
Benchmark 2: hyper_threading_pr threads: 2
Time (mean ± σ): 13.801 s ± 0.008 s [User: 25.580 s, System: 0.756 s]
Range (min … max): 13.795 s … 13.807 s 2 runs
Summary
'hyper_threading_main threads: 2' ran
1.00 ± 0.00 times faster than 'hyper_threading_pr threads: 2'
hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
Time (mean ± σ): 10.574 s ± 0.010 s [User: 38.631 s, System: 0.853 s]
Range (min … max): 10.567 s … 10.581 s 2 runs
Benchmark 2: hyper_threading_pr threads: 4
Time (mean ± σ): 10.529 s ± 0.075 s [User: 38.580 s, System: 0.897 s]
Range (min … max): 10.476 s … 10.582 s 2 runs
Summary
'hyper_threading_pr threads: 4' ran
1.00 ± 0.01 times faster than 'hyper_threading_main threads: 4'
hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
Time (mean ± σ): 10.592 s ± 0.227 s [User: 38.668 s, System: 0.856 s]
Range (min … max): 10.431 s … 10.752 s 2 runs
Benchmark 2: hyper_threading_pr threads: 6
Time (mean ± σ): 10.619 s ± 0.203 s [User: 38.835 s, System: 0.966 s]
Range (min … max): 10.475 s … 10.762 s 2 runs
Summary
'hyper_threading_main threads: 6' ran
1.00 ± 0.03 times faster than 'hyper_threading_pr threads: 6'
hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
Time (mean ± σ): 10.567 s ± 0.044 s [User: 38.750 s, System: 1.006 s]
Range (min … max): 10.536 s … 10.599 s 2 runs
Benchmark 2: hyper_threading_pr threads: 8
Time (mean ± σ): 10.383 s ± 0.064 s [User: 39.141 s, System: 0.911 s]
Range (min … max): 10.338 s … 10.428 s 2 runs
Summary
'hyper_threading_pr threads: 8' ran
1.02 ± 0.01 times faster than 'hyper_threading_main threads: 8'
hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
Time (mean ± σ): 10.563 s ± 0.205 s [User: 39.398 s, System: 0.991 s]
Range (min … max): 10.418 s … 10.707 s 2 runs
Benchmark 2: hyper_threading_pr threads: 16
Time (mean ± σ): 10.518 s ± 0.130 s [User: 39.375 s, System: 0.971 s]
Range (min … max): 10.426 s … 10.610 s 2 runs
Summary
'hyper_threading_pr threads: 16' ran
1.00 ± 0.02 times faster than 'hyper_threading_main threads: 16'
Attention: Patch coverage is 56.25000%
with 7 lines
in your changes are missing coverage. Please review.
Project coverage is 94.79%. Comparing base (
c5839fd
) to head (c9bbbf9
).
Files | Patch % | Lines |
---|---|---|
cairo-vm-cli/src/main.rs | 56.25% | 7 Missing :warning: |
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Benchmark Results for unmodified programs :rocket:
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base big_factorial |
2.074 ± 0.025 | 2.040 | 2.110 | 1.00 ± 0.01 |
head big_factorial |
2.073 ± 0.013 | 2.055 | 2.099 | 1.00 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base big_fibonacci |
2.088 ± 0.034 | 2.048 | 2.153 | 1.02 ± 0.02 |
head big_fibonacci |
2.053 ± 0.030 | 2.018 | 2.109 | 1.00 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base blake2s_integration_benchmark |
7.070 ± 0.091 | 6.940 | 7.207 | 1.00 |
head blake2s_integration_benchmark |
7.084 ± 0.068 | 6.998 | 7.175 | 1.00 ± 0.02 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base compare_arrays_200000 |
2.130 ± 0.027 | 2.098 | 2.180 | 1.01 ± 0.01 |
head compare_arrays_200000 |
2.115 ± 0.013 | 2.099 | 2.142 | 1.00 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base dict_integration_benchmark |
1.402 ± 0.025 | 1.375 | 1.459 | 1.01 ± 0.02 |
head dict_integration_benchmark |
1.391 ± 0.018 | 1.371 | 1.433 | 1.00 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base field_arithmetic_get_square_benchmark |
1.237 ± 0.015 | 1.223 | 1.274 | 1.00 |
head field_arithmetic_get_square_benchmark |
1.257 ± 0.030 | 1.228 | 1.318 | 1.02 ± 0.03 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base integration_builtins |
7.119 ± 0.091 | 7.009 | 7.240 | 1.00 |
head integration_builtins |
7.143 ± 0.076 | 7.066 | 7.304 | 1.00 ± 0.02 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base keccak_integration_benchmark |
7.288 ± 0.117 | 7.171 | 7.556 | 1.00 |
head keccak_integration_benchmark |
7.298 ± 0.103 | 7.235 | 7.567 | 1.00 ± 0.02 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base linear_search |
2.163 ± 0.033 | 2.132 | 2.231 | 1.00 |
head linear_search |
2.180 ± 0.052 | 2.152 | 2.320 | 1.01 ± 0.03 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base math_cmp_and_pow_integration_benchmark |
1.784 ± 0.013 | 1.766 | 1.804 | 1.00 |
head math_cmp_and_pow_integration_benchmark |
1.806 ± 0.058 | 1.764 | 1.934 | 1.01 ± 0.03 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base math_integration_benchmark |
1.582 ± 0.015 | 1.566 | 1.610 | 1.00 ± 0.01 |
head math_integration_benchmark |
1.576 ± 0.011 | 1.563 | 1.599 | 1.00 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base memory_integration_benchmark |
1.225 ± 0.023 | 1.200 | 1.275 | 1.01 ± 0.02 |
head memory_integration_benchmark |
1.211 ± 0.008 | 1.200 | 1.226 | 1.00 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base operations_with_data_structures_benchmarks |
1.855 ± 0.016 | 1.831 | 1.886 | 1.01 ± 0.01 |
head operations_with_data_structures_benchmarks |
1.844 ± 0.013 | 1.832 | 1.873 | 1.00 |
Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
---|---|---|---|---|
base pedersen |
531.9 ± 5.8 | 527.5 | 548.0 | 1.01 ± 0.01 |
head pedersen |
526.8 ± 1.1 | 525.2 | 528.4 | 1.00 |
Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
---|---|---|---|---|
base poseidon_integration_benchmark |
972.0 ± 9.5 | 962.6 | 987.6 | 1.00 |
head poseidon_integration_benchmark |
975.1 ± 12.5 | 963.7 | 1007.9 | 1.00 ± 0.02 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base secp_integration_benchmark |
1.857 ± 0.012 | 1.841 | 1.879 | 1.01 ± 0.01 |
head secp_integration_benchmark |
1.847 ± 0.008 | 1.839 | 1.861 | 1.00 |
Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
---|---|---|---|---|
base set_integration_benchmark |
694.3 ± 5.1 | 688.6 | 703.5 | 1.00 |
head set_integration_benchmark |
728.6 ± 5.6 | 724.0 | 743.8 | 1.05 ± 0.01 |
Command | Mean [s] | Min [s] | Max [s] | Relative |
---|---|---|---|---|
base uint256_integration_benchmark |
4.054 ± 0.027 | 4.012 | 4.086 | 1.00 |
head uint256_integration_benchmark |
4.058 ± 0.016 | 4.041 | 4.090 | 1.00 ± 0.01 |
Depends on #1729 & #1720 Adds
--run_from_cairo_pie
tocairo-vm-cli
Adds a workflow that runs all test programs's pies and compares the trace, memory and pie outputs against cairo_lang