search

ithacaxyz / odyssey

A testnet open-source Layer 2 from the future, co-designed with the developer tools stack.
https://www.ithaca.xyz/updates/odyssey
Apache License 2.0
200 stars 49 forks source link

feat: optimize and refactor forks #31

Closed malik672 closed 1 month ago

malik672 commented 1 month ago

Optmized

https://youtu.be/78Y17hAo96I?si=39DZpQIW_TwZicqB

malik672 commented 1 month ago

Please provide a benchmark, I am not convinced this is actually an optimization

I will add the benchmarks then request changes then after remove everything that has to do with the benchmark from the codebase

onbjerg commented 1 month ago

Why remove the benchmark afterwards?

malik672 commented 1 month ago

Why remove the benchmark afterwards?

It was not there initially so I thought maybe it's intentional

malik672 commented 1 month ago

Why remove the benchmark afterwards?

from 129 ns to 11 ns

onbjerg commented 1 month ago

Where is the benchmark?:)

malik672 commented 1 month ago

Where is the benchmark?:)

still on my pc :), I forgot to mention not the same logic as the pr

onbjerg commented 1 month ago

please push the benchmark as well

malik672 commented 1 month ago

please push the benchmark as well

Pushed the wrong code, give me 2 min

onbjerg commented 1 month ago

how do I run the benchmark? cargo bench -p odyssey-node does nto run anything

malik672 commented 1 month ago

how do I run the benchmark? cargo bench -p odyssey-node does not run anything

I only made the benchmark available in the node crate, so it will have to be cd crates, cd node , cargo bench

onbjerg commented 1 month ago

When I remove the rev which is incorrect, the benchmark marks this as worse in the cases that matter more in a live node, i.e. when you approach the latest fork (as we'll spend most of our live time on the latest fork)

Gnuplot not found, using plotters backend
revm_spec/original_block_1
                        time:   [141.28 ns 141.67 ns 142.13 ns]
                        change: [-8.7843% -8.3520% -7.9323%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 22 outliers among 100 measurements (22.00%)
  14 (14.00%) low severe
  7 (7.00%) low mild
  1 (1.00%) high severe
Benchmarking revm_spec/original_block_1000000: Collecting 100 samples in estimated 10.001 s (70M iteratiorevm_spec/original_block_1000000
                        time:   [144.28 ns 144.75 ns 145.26 ns]
                        change: [-6.7479% -6.4454% -6.1346%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking revm_spec/original_block_10000000: Collecting 100 samples in estimated 10.000 s (108M iteratrevm_spec/original_block_10000000
                        time:   [89.994 ns 90.283 ns 90.578 ns]
                        change: [-12.685% -12.299% -11.941%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking revm_spec/original_block_100000000: Collecting 100 samples in estimated 10.000 s (177M iterarevm_spec/original_block_100000000
                        time:   [56.280 ns 56.520 ns 56.762 ns]
                        change: [-20.501% -19.940% -19.343%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking revm_spec_opt/optimized_block_1: Collecting 100 samples in estimated 10.000 s (83M iterationrevm_spec_opt/optimized_block_1
                        time:   [121.85 ns 122.30 ns 122.69 ns]
                        change: [+1316.6% +1321.3% +1325.5%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking revm_spec_opt/optimized_block_1000000: Collecting 100 samples in estimated 10.000 s (83M iterevm_spec_opt/optimized_block_1000000
                        time:   [120.54 ns 120.90 ns 121.30 ns]
                        change: [+1309.8% +1312.4% +1315.1%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe
Benchmarking revm_spec_opt/optimized_block_10000000: Collecting 100 samples in estimated 10.000 s (126M irevm_spec_opt/optimized_block_10000000
                        time:   [79.367 ns 79.641 ns 79.930 ns]
                        change: [+824.02% +826.36% +828.63%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  8 (8.00%) high mild
  3 (3.00%) high severe
Benchmarking revm_spec_opt/optimized_block_100000000: Collecting 100 samples in estimated 10.000 s (169M revm_spec_opt/optimized_block_100000000
                        time:   [58.786 ns 58.901 ns 59.050 ns]
                        change: [+596.50% +598.41% +600.36%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

For that reason, and because I personally find this less readable, I'll close this PR. I appreciate your contribution nonetheless

malik672 commented 1 month ago

When I remove the rev which is incorrect, the benchmark marks this as worse in the cases that matter more in a live node, i.e. when you approach the latest fork (as we'll spend most of our live time on the latest fork)

Gnuplot not found, using plotters backend
revm_spec/original_block_1
                        time:   [141.28 ns 141.67 ns 142.13 ns]
                        change: [-8.7843% -8.3520% -7.9323%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 22 outliers among 100 measurements (22.00%)
  14 (14.00%) low severe
  7 (7.00%) low mild
  1 (1.00%) high severe
Benchmarking revm_spec/original_block_1000000: Collecting 100 samples in estimated 10.001 s (70M iteratiorevm_spec/original_block_1000000
                        time:   [144.28 ns 144.75 ns 145.26 ns]
                        change: [-6.7479% -6.4454% -6.1346%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking revm_spec/original_block_10000000: Collecting 100 samples in estimated 10.000 s (108M iteratrevm_spec/original_block_10000000
                        time:   [89.994 ns 90.283 ns 90.578 ns]
                        change: [-12.685% -12.299% -11.941%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking revm_spec/original_block_100000000: Collecting 100 samples in estimated 10.000 s (177M iterarevm_spec/original_block_100000000
                        time:   [56.280 ns 56.520 ns 56.762 ns]
                        change: [-20.501% -19.940% -19.343%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking revm_spec_opt/optimized_block_1: Collecting 100 samples in estimated 10.000 s (83M iterationrevm_spec_opt/optimized_block_1
                        time:   [121.85 ns 122.30 ns 122.69 ns]
                        change: [+1316.6% +1321.3% +1325.5%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking revm_spec_opt/optimized_block_1000000: Collecting 100 samples in estimated 10.000 s (83M iterevm_spec_opt/optimized_block_1000000
                        time:   [120.54 ns 120.90 ns 121.30 ns]
                        change: [+1309.8% +1312.4% +1315.1%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe
Benchmarking revm_spec_opt/optimized_block_10000000: Collecting 100 samples in estimated 10.000 s (126M irevm_spec_opt/optimized_block_10000000
                        time:   [79.367 ns 79.641 ns 79.930 ns]
                        change: [+824.02% +826.36% +828.63%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  8 (8.00%) high mild
  3 (3.00%) high severe
Benchmarking revm_spec_opt/optimized_block_100000000: Collecting 100 samples in estimated 10.000 s (169M revm_spec_opt/optimized_block_100000000
                        time:   [58.786 ns 58.901 ns 59.050 ns]
                        change: [+596.50% +598.41% +600.36%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

For that reason, and because I personally find this less readable, I'll close this PR. I appreciate your contribution nonetheless

The rev is the main optimization, I mean rev or no rev it should give the same result right :)