Serialize `Array<Felt252>` return value into output segment

fmoletta commented 1 month ago

Checks that only PanicResult<Array<Felt252>> or Array<Felt252> can be returned by the program when running with either --proof_mode or --append_return_values.
Serializes return values under the previous conditions following the format: PanicResult<Array<Felt252>> -> [panic_flag, array_len, arr[0], arr[1],.., arr[n]] t<Array<Felt252> -> [array_len, arr[0], arr[1],.., arr[n]]
Creates mirror versions of all test programs modified to return a serialized output so we can keep the same level of testing

github-actions[bot] commented 1 month ago

**Hyper Thereading Benchmark results**

hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     27.162 s ±  0.008 s    [User: 26.367 s, System: 0.794 s]
  Range (min … max):   27.156 s … 27.168 s    2 runs

Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     27.250 s ±  0.033 s    [User: 26.412 s, System: 0.836 s]
  Range (min … max):   27.226 s … 27.273 s    2 runs

Summary
  'hyper_threading_main threads: 1' ran
    1.00 ± 0.00 times faster than 'hyper_threading_pr threads: 1'

hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     14.794 s ±  0.010 s    [User: 26.924 s, System: 0.830 s]
  Range (min … max):   14.787 s … 14.800 s    2 runs

Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     14.839 s ±  0.042 s    [User: 27.001 s, System: 0.798 s]
  Range (min … max):   14.810 s … 14.869 s    2 runs

Summary
  'hyper_threading_main threads: 2' ran
    1.00 ± 0.00 times faster than 'hyper_threading_pr threads: 2'

hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):     10.585 s ±  0.158 s    [User: 39.043 s, System: 1.019 s]
  Range (min … max):   10.473 s … 10.696 s    2 runs

Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):     10.710 s ±  0.146 s    [User: 39.029 s, System: 1.039 s]
  Range (min … max):   10.607 s … 10.813 s    2 runs

Summary
  'hyper_threading_main threads: 4' ran
    1.01 ± 0.02 times faster than 'hyper_threading_pr threads: 4'

hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):     10.790 s ±  0.046 s    [User: 39.260 s, System: 0.994 s]
  Range (min … max):   10.758 s … 10.823 s    2 runs

Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):     10.888 s ±  0.002 s    [User: 39.034 s, System: 1.027 s]
  Range (min … max):   10.887 s … 10.889 s    2 runs

Summary
  'hyper_threading_main threads: 6' ran
    1.01 ± 0.00 times faster than 'hyper_threading_pr threads: 6'

hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):     10.725 s ±  0.022 s    [User: 39.408 s, System: 1.011 s]
  Range (min … max):   10.709 s … 10.741 s    2 runs

Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):     10.558 s ±  0.236 s    [User: 39.257 s, System: 0.994 s]
  Range (min … max):   10.391 s … 10.725 s    2 runs

Summary
  'hyper_threading_pr threads: 8' ran
    1.02 ± 0.02 times faster than 'hyper_threading_main threads: 8'

hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):     10.723 s ±  0.283 s    [User: 39.682 s, System: 1.116 s]
  Range (min … max):   10.523 s … 10.923 s    2 runs

Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):     10.743 s ±  0.417 s    [User: 39.696 s, System: 1.155 s]
  Range (min … max):   10.448 s … 11.038 s    2 runs

Summary
  'hyper_threading_main threads: 16' ran
    1.00 ± 0.05 times faster than 'hyper_threading_pr threads: 16'

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 92.61364% with 13 lines in your changes are missing coverage. Please review.

Project coverage is 94.79%. Comparing base (79cb881) to head (bfc542f).

Files	Patch %	Lines
cairo1-run/src/cairo_run.rs	92.57%	13 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #1764 +/- ## ========================================== - Coverage 94.79% 94.79% -0.01% ========================================== Files 101 101 Lines 38787 38889 +102 ========================================== + Hits 36770 36863 +93 - Misses 2017 2026 +9 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 1 month ago

Benchmark Results for unmodified programs :rocket:

Command	Mean [s]	Min [s]	Max [s]	Relative
`base big_factorial`	2.001 ± 0.020	1.983	2.052	1.00
`head big_factorial`	2.012 ± 0.028	1.979	2.052	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base big_fibonacci`	1.985 ± 0.054	1.941	2.123	1.00
`head big_fibonacci`	2.007 ± 0.148	1.933	2.422	1.01 ± 0.08

Command	Mean [s]	Min [s]	Max [s]	Relative
`base blake2s_integration_benchmark`	7.493 ± 0.091	7.381	7.661	1.00
`head blake2s_integration_benchmark`	7.531 ± 0.159	7.385	7.841	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base compare_arrays_200000`	2.110 ± 0.039	2.080	2.201	1.00
`head compare_arrays_200000`	2.124 ± 0.025	2.093	2.159	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base dict_integration_benchmark`	1.412 ± 0.025	1.393	1.477	1.00
`head dict_integration_benchmark`	1.415 ± 0.017	1.396	1.449	1.00 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base field_arithmetic_get_square_benchmark`	1.286 ± 0.031	1.259	1.361	1.00 ± 0.03
`head field_arithmetic_get_square_benchmark`	1.283 ± 0.015	1.261	1.300	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base integration_builtins`	7.520 ± 0.102	7.387	7.689	1.00
`head integration_builtins`	7.558 ± 0.127	7.403	7.724	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base keccak_integration_benchmark`	7.724 ± 0.108	7.609	7.915	1.00
`head keccak_integration_benchmark`	7.751 ± 0.125	7.646	8.025	1.00 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base linear_search`	2.066 ± 0.030	2.039	2.121	1.01 ± 0.02
`head linear_search`	2.039 ± 0.008	2.027	2.056	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base math_cmp_and_pow_integration_benchmark`	1.682 ± 0.007	1.675	1.700	1.00 ± 0.01
`head math_cmp_and_pow_integration_benchmark`	1.677 ± 0.008	1.668	1.696	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base math_integration_benchmark`	1.596 ± 0.008	1.582	1.606	1.00
`head math_integration_benchmark`	1.602 ± 0.023	1.583	1.647	1.00 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base memory_integration_benchmark`	1.191 ± 0.021	1.177	1.246	1.00
`head memory_integration_benchmark`	1.197 ± 0.020	1.179	1.243	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base operations_with_data_structures_benchmarks`	1.802 ± 0.012	1.790	1.827	1.00
`head operations_with_data_structures_benchmarks`	1.811 ± 0.016	1.798	1.846	1.00 ± 0.01

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`base pedersen`	513.5 ± 2.8	510.5	517.3	1.00 ± 0.01
`head pedersen`	512.4 ± 3.0	509.6	518.6	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`base poseidon_integration_benchmark`	958.6 ± 7.5	950.3	972.7	1.00
`head poseidon_integration_benchmark`	965.5 ± 11.7	953.7	986.9	1.01 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
`base secp_integration_benchmark`	1.867 ± 0.011	1.845	1.880	1.01 ± 0.01
`head secp_integration_benchmark`	1.853 ± 0.020	1.834	1.893	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`base set_integration_benchmark`	637.8 ± 2.1	635.8	642.7	1.00
`head set_integration_benchmark`	640.2 ± 2.1	638.1	643.7	1.00 ± 0.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base uint256_integration_benchmark`	4.142 ± 0.051	4.083	4.226	1.00
`head uint256_integration_benchmark`	4.244 ± 0.084	4.122	4.385	1.02 ± 0.02

lambdaclass / cairo-vm

Serialize `Array<Felt252>` return value into output segment #1764

Codecov Report