search

lambdaclass / cairo-vm

cairo-vm is a Rust implementation of the Cairo VM. Cairo (CPU Algebraic Intermediate Representation) is a programming language for writing provable programs, where one party can prove to another that a certain computation was executed correctly without the need for this party to re-execute the same program.
https://lambdaclass.github.io/cairo-vm
Apache License 2.0
485 stars 133 forks source link

Add option to pass Sierra programs to cairo1-run #1719

Closed fmoletta closed 2 months ago

fmoletta commented 2 months ago

Mirrors #1544 + Adds some fixes

github-actions[bot] commented 2 months ago

Benchmark Results for unmodified programs :rocket:

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 1.942 ± 0.045 1.871 2.013 1.01 ± 0.03
head big_factorial 1.930 ± 0.023 1.876 1.958 1.00
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 1.880 ± 0.095 1.821 2.129 1.01 ± 0.06
head big_fibonacci 1.871 ± 0.042 1.816 1.927 1.00
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 6.447 ± 0.117 6.300 6.616 1.00
head blake2s_integration_benchmark 6.509 ± 0.168 6.328 6.891 1.01 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 1.915 ± 0.053 1.873 2.044 1.00
head compare_arrays_200000 1.932 ± 0.091 1.878 2.179 1.01 ± 0.06
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 1.339 ± 0.044 1.287 1.418 1.02 ± 0.04
head dict_integration_benchmark 1.310 ± 0.016 1.290 1.340 1.00
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.158 ± 0.025 1.138 1.223 1.00
head field_arithmetic_get_square_benchmark 1.174 ± 0.026 1.138 1.222 1.01 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 6.518 ± 0.113 6.376 6.663 1.00
head integration_builtins 6.619 ± 0.115 6.394 6.832 1.02 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 6.700 ± 0.116 6.520 6.867 1.01 ± 0.02
head keccak_integration_benchmark 6.623 ± 0.076 6.551 6.754 1.00
Command Mean [s] Min [s] Max [s] Relative
base linear_search 1.924 ± 0.035 1.887 1.987 1.00
head linear_search 1.935 ± 0.032 1.892 1.994 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 1.699 ± 0.038 1.654 1.764 1.02 ± 0.03
head math_cmp_and_pow_integration_benchmark 1.671 ± 0.019 1.644 1.699 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 1.488 ± 0.043 1.439 1.536 1.01 ± 0.04
head math_integration_benchmark 1.471 ± 0.040 1.439 1.557 1.00
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 1.134 ± 0.020 1.106 1.169 1.01 ± 0.02
head memory_integration_benchmark 1.123 ± 0.013 1.110 1.143 1.00
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 1.755 ± 0.021 1.735 1.796 1.00
head operations_with_data_structures_benchmarks 1.762 ± 0.018 1.745 1.801 1.00 ± 0.02
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 501.8 ± 3.3 498.0 505.7 1.00
head pedersen 502.5 ± 4.8 496.2 510.9 1.00 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base poseidon_integration_benchmark 945.2 ± 11.7 929.9 973.0 1.00
head poseidon_integration_benchmark 947.5 ± 6.7 937.1 958.6 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 1.740 ± 0.031 1.717 1.813 1.00
head secp_integration_benchmark 1.753 ± 0.013 1.735 1.773 1.01 ± 0.02
Command Mean [ms] Min [ms] Max [ms] Relative
base set_integration_benchmark 708.6 ± 15.7 697.7 752.1 1.01 ± 0.02
head set_integration_benchmark 705.0 ± 5.0 699.0 715.7 1.00
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 3.757 ± 0.077 3.681 3.931 1.01 ± 0.03
head uint256_integration_benchmark 3.734 ± 0.055 3.679 3.819 1.00
codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 88.88889% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 94.79%. Comparing base (7796f24) to head (dbd5895).

Files Patch % Lines
cairo1-run/src/main.rs 88.88% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1719 +/- ## ========================================== - Coverage 94.80% 94.79% -0.01% ========================================== Files 100 100 Lines 38212 38213 +1 ========================================== Hits 36225 36225 - Misses 1987 1988 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 2 months ago 
**Hyper Thereading Benchmark results**

hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     23.641 s ±  0.078 s    [User: 22.924 s, System: 0.716 s]
  Range (min … max):   23.587 s … 23.696 s    2 runs

Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     23.556 s ±  0.014 s    [User: 22.812 s, System: 0.742 s]
  Range (min … max):   23.546 s … 23.565 s    2 runs

Summary
  'hyper_threading_pr threads: 1' ran
    1.00 ± 0.00 times faster than 'hyper_threading_main threads: 1'

hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     12.310 s ±  0.024 s    [User: 23.261 s, System: 0.744 s]
  Range (min … max):   12.294 s … 12.327 s    2 runs

Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     12.265 s ±  0.004 s    [User: 23.202 s, System: 0.674 s]
  Range (min … max):   12.263 s … 12.268 s    2 runs

Summary
  'hyper_threading_pr threads: 2' ran
    1.00 ± 0.00 times faster than 'hyper_threading_main threads: 2'

hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):     10.023 s ±  0.223 s    [User: 35.007 s, System: 0.949 s]
  Range (min … max):    9.865 s … 10.181 s    2 runs

Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):      9.974 s ±  0.164 s    [User: 35.085 s, System: 0.918 s]
  Range (min … max):    9.858 s … 10.090 s    2 runs

Summary
  'hyper_threading_pr threads: 4' ran
    1.00 ± 0.03 times faster than 'hyper_threading_main threads: 4'

hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):      9.524 s ±  0.150 s    [User: 35.702 s, System: 0.964 s]
  Range (min … max):    9.418 s …  9.630 s    2 runs

Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):      9.758 s ±  0.001 s    [User: 35.501 s, System: 0.984 s]
  Range (min … max):    9.758 s …  9.759 s    2 runs

Summary
  'hyper_threading_main threads: 6' ran
    1.02 ± 0.02 times faster than 'hyper_threading_pr threads: 6'

hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):      9.582 s ±  0.014 s    [User: 35.619 s, System: 0.992 s]
  Range (min … max):    9.572 s …  9.592 s    2 runs

Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):      9.633 s ±  0.065 s    [User: 35.606 s, System: 0.967 s]
  Range (min … max):    9.586 s …  9.679 s    2 runs

Summary
  'hyper_threading_main threads: 8' ran
    1.01 ± 0.01 times faster than 'hyper_threading_pr threads: 8'

hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):      9.689 s ±  0.035 s    [User: 35.975 s, System: 1.007 s]
  Range (min … max):    9.665 s …  9.714 s    2 runs

Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):      9.662 s ±  0.106 s    [User: 35.814 s, System: 1.077 s]
  Range (min … max):    9.587 s …  9.737 s    2 runs

Summary
  'hyper_threading_pr threads: 16' ran
    1.00 ± 0.01 times faster than 'hyper_threading_main threads: 16'
fmoletta commented 2 months ago

Nice @fmoletta ! Just one question, I am getting an error when trying to execute .sierra programs

➜  cairo1-run git:(accept-sierra-cairo1-run) ✗ cargo run poseidon.json.sierra --layout all_cairo
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `/Users/peter/repos/cairo-vm/target/debug/cairo1-run poseidon.json.sierra --layout all_cairo`
Error: SierraCompilation("Only files with .cairo extension can be compiled.")

I compiled the sierra with cairo compiler v2.6.3

cargo run --bin cairo-compile -- --single-file ../cairo-vm/cairo_programs/cairo-1-programs/poseidon.cairo ../cairo-vm/cairo1-run/poseidon.json.sierra --replace-ids

Also, do you have a scarb project which I can test to execute?

scarb build and cairo-compile yield different compiled sierra file formats, thats why it fails. Creating a project using scarb init creates a scarb project with a fibonacci code

pefontana commented 2 months ago

Nice @fmoletta ! Just one question, I am getting an error when trying to execute .sierra programs

➜  cairo1-run git:(accept-sierra-cairo1-run) ✗ cargo run poseidon.json.sierra --layout all_cairo
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `/Users/peter/repos/cairo-vm/target/debug/cairo1-run poseidon.json.sierra --layout all_cairo`
Error: SierraCompilation("Only files with .cairo extension can be compiled.")

I compiled the sierra with cairo compiler v2.6.3

cargo run --bin cairo-compile -- --single-file ../cairo-vm/cairo_programs/cairo-1-programs/poseidon.cairo ../cairo-vm/cairo1-run/poseidon.json.sierra --replace-ids

Also, do you have a scarb project which I can test to execute?

scarb build and cairo-compile yield different compiled sierra file formats, thats why it fails. Creating a project using scarb init creates a scarb project with a fibonacci code

Great! Thanks fede