paritytech / reed-solomon-novelpoly

Novel polynomial basis for a reed solomon encoder
30 stars 6 forks source link

chores: bump to edition 2021, update naiive crate dep, update criterion, fix `fn` ident #7

Closed drahnr closed 1 year ago

drahnr commented 1 year ago

Also fixes a compilation issue, besides the chores.


Benchmarking outcome (cargo criterion) with rustc 1.69.0 (84c898d65 2023-04-16)

# cpuinfo
..
Brand Raw: AMD Ryzen 9 5950X 16-Core Processor
Hz Advertised Friendly: 2.2000 GHz
Hz Actual Friendly: 2.2000 GHz
Hz Advertised: (2200000000, 0)
Hz Actual: (2200000000, 0)
Arch: X86_64
Bits: 64
Count: 32
Arch String Raw: x86_64
L1 Data Cache Size: 524288
L1 Instruction Cache Size: 524288
L2 Cache Size: 8388608
L2 Cache Line Size: 512
L2 Cache Associativity: 6
L3 Cache Size: 524288
..
Benchmarking parameterized encode validator_count=4/novel-poly-encode/1000: Collecting 10 samples in estimated 5.0006 s (293k                                                                                                                               parameterized encode validator_count=4/novel-poly-encode/1000                        
                        time:   [16.722 µs 16.963 µs 17.150 µs]
                        change: [+0.5241% +2.0545% +3.8009%] (p = 0.04 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/1112000: Collecting 10 samples in estimated 5.1209 s (22                                                                                                                              parameterized encode validator_count=4/novel-poly-encode/1112000                        
                        time:   [19.322 ms 19.518 ms 19.685 ms]
                        change: [+5.2683% +6.0682% +6.7714%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/2223000: Collecting 10 samples in estimated 7.2301 s (16                                                                                                                              parameterized encode validator_count=4/novel-poly-encode/2223000                        
                        time:   [37.033 ms 37.366 ms 37.921 ms]
                        change: [+0.0520% +1.0367% +2.1919%] (p = 0.09 > 0.05)
                        No change in performance detected.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/3334000: Collecting 10 samples in estimated 6.5799 s (11                                                                                                                              parameterized encode validator_count=4/novel-poly-encode/3334000                        
                        time:   [57.612 ms 58.083 ms 58.632 ms]
                        change: [-0.2785% +0.8038% +2.0830%] (p = 0.22 > 0.05)
                        No change in performance detected.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/4445000: Collecting 10 samples in estimated 9.4046 s (11                                                                                                                              parameterized encode validator_count=4/novel-poly-encode/4445000                        
                        time:   [77.552 ms 77.965 ms 78.540 ms]
                        change: [+1.1110% +1.8591% +2.6149%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/5556000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.8s or enable flat sampling.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/5556000: Collecting 10 samples in estimated 6.8484 s (55                                                                                                                              parameterized encode validator_count=4/novel-poly-encode/5556000                        
                        time:   [96.945 ms 97.266 ms 97.532 ms]
                        change: [-2.4814% -1.9861% -1.5898%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/6667000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 7.9s or enable flat sampling.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/6667000: Collecting 10 samples in estimated 7.9028 s (55                                                                                                                              parameterized encode validator_count=4/novel-poly-encode/6667000                        
                        time:   [115.93 ms 116.13 ms 116.39 ms]
                        change: [-2.4880% -2.1258% -1.7894%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/7778000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.0s or enable flat sampling.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/7778000: Collecting 10 samples in estimated 9.0187 s (55                                                                                                                              parameterized encode validator_count=4/novel-poly-encode/7778000                        
                        time:   [136.52 ms 136.78 ms 137.04 ms]
                        change: [-0.4451% -0.2491% -0.0643%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/8889000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.6s or enable flat sampling.
Benchmarking parameterized encode validator_count=4/novel-poly-encode/8889000: Collecting 10 samples in estimated 9.5897 s (55                                                                                                                              parameterized encode validator_count=4/novel-poly-encode/8889000                        
                        time:   [157.35 ms 157.57 ms 157.98 ms]
                        change: [+0.7741% +1.9449% +3.2533%] (p = 0.00 < 0.05)
                        Change within noise threshold.

Benchmarking parameterized encode validator_count=36/novel-poly-encode/1000: Collecting 10 samples in estimated 5.0011 s (89k                                                                                                                               parameterized encode validator_count=36/novel-poly-encode/1000                        
                        time:   [45.528 µs 45.797 µs 45.928 µs]
                        change: [-3.1655% -2.2019% -1.3332%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/1112000: Collecting 10 samples in estimated 6.0395 s (1                                                                                                                              parameterized encode validator_count=36/novel-poly-encode/1112000                        
                        time:   [49.856 ms 50.014 ms 50.093 ms]
                        change: [+2.3664% +2.7480% +3.1038%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/2223000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.4s or enable flat sampling.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/2223000: Collecting 10 samples in estimated 6.4015 s (5                                                                                                                              parameterized encode validator_count=36/novel-poly-encode/2223000                        
                        time:   [98.148 ms 99.758 ms 100.76 ms]
                        change: [-5.0153% -0.8448% +2.0502%] (p = 0.77 > 0.05)
                        No change in performance detected.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/3334000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 8.2s or enable flat sampling.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/3334000: Collecting 10 samples in estimated 8.2420 s (5                                                                                                                              parameterized encode validator_count=36/novel-poly-encode/3334000                        
                        time:   [142.82 ms 143.05 ms 143.32 ms]
                        change: [-13.079% -10.629% -8.7149%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/4445000: Collecting 10 samples in estimated 6.4795 s (3                                                                                                                              parameterized encode validator_count=36/novel-poly-encode/4445000                        
                        time:   [189.68 ms 190.79 ms 192.15 ms]
                        change: [-15.914% -13.855% -11.858%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/5556000: Collecting 10 samples in estimated 5.5400 s (2                                                                                                                              parameterized encode validator_count=36/novel-poly-encode/5556000                        
                        time:   [241.83 ms 246.04 ms 249.97 ms]
                        change: [-15.387% -10.133% -5.1149%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/6667000: Collecting 10 samples in estimated 6.7533 s (2                                                                                                                              parameterized encode validator_count=36/novel-poly-encode/6667000                        
                        time:   [309.61 ms 310.57 ms 311.76 ms]
                        change: [-3.7559% -1.4396% +0.5581%] (p = 0.25 > 0.05)
                        No change in performance detected.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/7778000: Collecting 10 samples in estimated 7.6389 s (2                                                                                                                              parameterized encode validator_count=36/novel-poly-encode/7778000                        
                        time:   [354.49 ms 361.17 ms 367.70 ms]
                        change: [-3.4939% -0.7453% +1.8371%] (p = 0.60 > 0.05)
                        No change in performance detected.
Benchmarking parameterized encode validator_count=36/novel-poly-encode/8889000: Collecting 10 samples in estimated 8.4624 s (2                                                                                                                              parameterized encode validator_count=36/novel-poly-encode/8889000                        
                        time:   [411.75 ms 416.14 ms 420.64 ms]
                        change: [-1.2526% +0.1072% +1.5363%] (p = 0.89 > 0.05)
                        No change in performance detected.

Benchmarking parameterized encode validator_count=68/novel-poly-encode/1000: Collecting 10 samples in estimated 5.0027 s (93k                                                                                                                               parameterized encode validator_count=68/novel-poly-encode/1000                        
                        time:   [46.175 µs 46.338 µs 46.560 µs]
                        change: [-3.5639% -2.8518% -2.1735%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/1112000: Collecting 10 samples in estimated 6.3791 s (1                                                                                                                              parameterized encode validator_count=68/novel-poly-encode/1112000                        
                        time:   [49.436 ms 49.507 ms 49.619 ms]
                        change: [+0.9403% +1.3299% +1.7242%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/2223000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.8s or enable flat sampling.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/2223000: Collecting 10 samples in estimated 6.7596 s (5                                                                                                                              parameterized encode validator_count=68/novel-poly-encode/2223000                        
                        time:   [93.588 ms 94.620 ms 95.911 ms]
                        change: [+3.7973% +5.3233% +6.8524%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/3334000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.0s or enable flat sampling.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/3334000: Collecting 10 samples in estimated 9.0062 s (55 iterations)^R
                                                                                                                              parameterized encode validator_count=68/novel-poly-encode/3334000                        
                        time:   [145.75 ms 148.85 ms 153.16 ms]
                        change: [+3.0856% +4.4821% +5.7710%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/4445000: Collecting 10 samples in estimated 6.7432 s (3                                                                                                                              parameterized encode validator_count=68/novel-poly-encode/4445000                        
                        time:   [200.00 ms 200.84 ms 201.80 ms]
                        change: [+3.3519% +3.7826% +4.2490%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/5556000: Collecting 10 samples in estimated 5.2248 s (2                                                                                                                              parameterized encode validator_count=68/novel-poly-encode/5556000                        
                        time:   [242.64 ms 243.07 ms 243.60 ms]
                        change: [+0.7049% +0.9216% +1.1581%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/6667000: Collecting 10 samples in estimated 6.7614 s (2                                                                                                                              parameterized encode validator_count=68/novel-poly-encode/6667000                        
                        time:   [306.07 ms 307.30 ms 308.46 ms]
                        change: [+5.4876% +5.9848% +6.4633%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/7778000: Collecting 10 samples in estimated 7.6280 s (2                                                                                                                              parameterized encode validator_count=68/novel-poly-encode/7778000                        
                        time:   [358.76 ms 360.55 ms 362.24 ms]
                        change: [+6.2508% +7.3632% +8.5280%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode validator_count=68/novel-poly-encode/8889000: Collecting 10 samples in estimated 8.2244 s (2                                                                                                                              parameterized encode validator_count=68/novel-poly-encode/8889000                        
                        time:   [391.46 ms 397.25 ms 403.24 ms]
                        change: [+3.9687% +6.0201% +7.9434%] (p = 0.00 < 0.05)
                        Performance has regressed.

Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/1000: Collecting 10 samples in estimated 5.000                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/1000                        
                        time:   [239.71 ns 240.80 ns 242.18 ns]
                        change: [-2.0552% -1.4975% -0.8930%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/1112000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/1112000                        
                        time:   [39.200 µs 39.650 µs 40.206 µs]
                        change: [-3.0420% -2.3719% -1.5411%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/2223000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/2223000                        
                        time:   [76.361 µs 77.501 µs 78.470 µs]
                        change: [-2.7765% -1.6878% -0.3629%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/3334000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/3334000                        
                        time:   [116.97 µs 118.91 µs 120.30 µs]
                        change: [-4.2815% -2.3385% -0.1685%] (p = 0.06 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/4445000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/4445000                        
                        time:   [164.34 µs 166.60 µs 168.37 µs]
                        change: [+2.1989% +4.1996% +6.5536%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/5556000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/5556000                        
                        time:   [225.09 µs 228.39 µs 232.24 µs]
                        change: [+3.3934% +6.0691% +9.0504%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/6667000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/6667000                        
                        time:   [316.39 µs 320.42 µs 323.10 µs]
                        change: [-0.5720% +2.7308% +5.8608%] (p = 0.12 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/7778000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/7778000                        
                        time:   [2.2722 ms 2.2900 ms 2.3265 ms]
                        change: [+416.77% +430.28% +445.18%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/8889000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/8889000                        
                        time:   [3.2475 ms 3.2885 ms 3.3559 ms]
                        change: [+31.146% +33.674% +36.508%] (p = 0.00 < 0.05)
                        Performance has regressed.

Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/1000: Collecting 10 samples in estimated 5.02                                                                                                                              parameterized reconstruct validator_count=36/novel-poly-reconstruct/1000                        
                        time:   [861.14 µs 863.36 µs 865.03 µs]
                        change: [+0.7473% +1.9859% +3.1206%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/1112000: Collecting 10 samples in estimated 8                                                                                                                              parameterized reconstruct validator_count=36/novel-poly-reconstruct/1112000                        
                        time:   [76.569 ms 78.296 ms 79.430 ms]
                        change: [+1.3487% +2.6240% +4.1985%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/2223000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 8.5s or enable flat sampling.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/2223000: Collecting 10 samples in estimated 8                                                                                                                              parameterized reconstruct validator_count=36/novel-poly-reconstruct/2223000                        
                        time:   [154.61 ms 154.96 ms 155.43 ms]
                        change: [+0.2407% +0.7631% +1.3913%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/3334000: Collecting 10 samples in estimated 6                                                                                                                              parameterized reconstruct validator_count=36/novel-poly-reconstruct/3334000                        
                        time:   [230.69 ms 233.32 ms 235.92 ms]
                        change: [-1.3023% +0.2524% +1.5953%] (p = 0.75 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/4445000: Collecting 10 samples in estimated 6                                                                                                                              parameterized reconstruct validator_count=36/novel-poly-reconstruct/4445000                        
                        time:   [313.63 ms 316.13 ms 318.63 ms]
                        change: [-1.0543% +0.3799% +1.8239%] (p = 0.63 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/5556000: Collecting 10 samples in estimated 7                                                                                                                              parameterized reconstruct validator_count=36/novel-poly-reconstruct/5556000                        
                        time:   [385.38 ms 387.41 ms 389.46 ms]
                        change: [-0.8567% -0.1112% +0.6436%] (p = 0.79 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/6667000: Collecting 10 samples in estimated 9                                                                                                                              parameterized reconstruct validator_count=36/novel-poly-reconstruct/6667000                        
                        time:   [465.21 ms 472.38 ms 479.11 ms]
                        change: [-1.3777% +0.7692% +2.9817%] (p = 0.51 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/7778000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 5.3s.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/7778000: Collecting 10 samples in estimated 5                                                                                                                              parameterized reconstruct validator_count=36/novel-poly-reconstruct/7778000                        
                        time:   [546.39 ms 555.86 ms 563.86 ms]
                        change: [-0.0845% +1.8509% +3.7129%] (p = 0.08 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/8889000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.7s.
Benchmarking parameterized reconstruct validator_count=36/novel-poly-reconstruct/8889000: Collecting 10 samples in estimated 6                                                                                                                              parameterized reconstruct validator_count=36/novel-poly-reconstruct/8889000                        
                        time:   [637.63 ms 640.95 ms 643.33 ms]
                        change: [+1.9814% +2.8450% +3.7888%] (p = 0.00 < 0.05)
                        Performance has regressed.

Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/1000: Collecting 10 samples in estimated 5.01                                                                                                                              parameterized reconstruct validator_count=68/novel-poly-reconstruct/1000                        
                        time:   [857.43 µs 864.19 µs 871.75 µs]
                        change: [-2.9070% -1.5187% +0.0505%] (p = 0.06 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/1112000: Collecting 10 samples in estimated 9                                                                                                                              parameterized reconstruct validator_count=68/novel-poly-reconstruct/1112000                        
                        time:   [85.425 ms 85.583 ms 85.658 ms]
                        change: [+1.2534% +2.2184% +3.1313%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/2223000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.4s or enable flat sampling.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/2223000: Collecting 10 samples in estimated 9                                                                                                                              parameterized reconstruct validator_count=68/novel-poly-reconstruct/2223000                        
                        time:   [162.13 ms 164.96 ms 168.54 ms]
                        change: [-1.8324% -0.6450% +0.3839%] (p = 0.33 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/3334000: Collecting 10 samples in estimated 5                                                                                                                              parameterized reconstruct validator_count=68/novel-poly-reconstruct/3334000                        
                        time:   [241.64 ms 245.60 ms 249.91 ms]
                        change: [-1.0837% +0.7288% +2.7102%] (p = 0.47 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/4445000: Collecting 10 samples in estimated 6                                                                                                                              parameterized reconstruct validator_count=68/novel-poly-reconstruct/4445000                        
                        time:   [330.50 ms 331.23 ms 331.87 ms]
                        change: [+0.6245% +1.1306% +1.7007%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/5556000: Collecting 10 samples in estimated 8                                                                                                                              parameterized reconstruct validator_count=68/novel-poly-reconstruct/5556000                        
                        time:   [405.54 ms 412.76 ms 420.21 ms]
                        change: [-0.5005% +1.2816% +3.2591%] (p = 0.24 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/6667000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 5.2s.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/6667000: Collecting 10 samples in estimated 5                                                                                                                              parameterized reconstruct validator_count=68/novel-poly-reconstruct/6667000                        
                        time:   [510.12 ms 513.50 ms 516.66 ms]
                        change: [+1.5013% +2.5678% +3.6595%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/7778000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.0s.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/7778000: Collecting 10 samples in estimated 5                                                                                                                              parameterized reconstruct validator_count=68/novel-poly-reconstruct/7778000                        
                        time:   [568.65 ms 574.79 ms 581.92 ms]
                        change: [-1.2400% +0.0279% +1.4927%] (p = 0.97 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/8889000: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.7s.
Benchmarking parameterized reconstruct validator_count=68/novel-poly-reconstruct/8889000: Collecting 10 samples in estimated 6                                                                                                                              parameterized reconstruct validator_count=68/novel-poly-reconstruct/8889000                        
                        time:   [650.95 ms 656.56 ms 661.59 ms]
                        change: [-0.2698% +0.6998% +1.6641%] (p = 0.20 > 0.05)
                        No change in performance detected.

Benchmarking parameterized encode fixed payload/novel-poly-encode/4: Collecting 10 samples in estimated 5.2719 s (275 iteratio                                                                                                                              parameterized encode fixed payload/novel-poly-encode/4                        
                        time:   [17.014 ms 17.030 ms 17.046 ms]
                        change: [-0.7759% -0.6211% -0.4414%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized encode fixed payload/novel-poly-encode/70: Collecting 10 samples in estimated 5.5904 s (110 iterati                                                                                                                              parameterized encode fixed payload/novel-poly-encode/70                        
                        time:   [43.630 ms 43.741 ms 44.033 ms]
                        change: [+3.4299% +6.0086% +9.6182%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode fixed payload/novel-poly-encode/136: Collecting 10 samples in estimated 5.3723 s (110 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/136                        
                        time:   [42.874 ms 42.976 ms 43.074 ms]
                        change: [+4.7800% +5.0857% +5.3466%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode fixed payload/novel-poly-encode/202: Collecting 10 samples in estimated 5.8206 s (220 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/202                        
                        time:   [24.325 ms 24.359 ms 24.404 ms]
                        change: [+2.4648% +3.1764% +3.9185%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode fixed payload/novel-poly-encode/268: Collecting 10 samples in estimated 6.1405 s (110 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/268                        
                        time:   [47.346 ms 47.558 ms 47.690 ms]
                        change: [+1.8391% +2.2574% +2.7154%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode fixed payload/novel-poly-encode/334: Collecting 10 samples in estimated 5.9116 s (110 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/334                        
                        time:   [48.421 ms 48.599 ms 48.772 ms]
                        change: [+7.2664% +7.8907% +8.5212%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode fixed payload/novel-poly-encode/400: Collecting 10 samples in estimated 5.5265 s (165 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/400                        
                        time:   [24.573 ms 24.634 ms 24.729 ms]
                        change: [+0.7166% +1.8292% +2.7818%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized encode fixed payload/novel-poly-encode/466: Collecting 10 samples in estimated 6.1715 s (220 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/466                        
                        time:   [25.799 ms 25.842 ms 25.895 ms]
                        change: [+1.7627% +2.0307% +2.2670%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode fixed payload/novel-poly-encode/532: Collecting 10 samples in estimated 6.3705 s (110 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/532                        
                        time:   [49.348 ms 49.439 ms 49.515 ms]
                        change: [+3.2120% +3.4550% +3.6709%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode fixed payload/novel-poly-encode/598: Collecting 10 samples in estimated 6.3244 s (110 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/598                        
                        time:   [48.825 ms 48.961 ms 49.060 ms]
                        change: [+0.2173% +0.4212% +0.6317%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized encode fixed payload/novel-poly-encode/664: Collecting 10 samples in estimated 6.1971 s (110 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/664                        
                        time:   [49.571 ms 49.782 ms 49.998 ms]
                        change: [+0.0496% +0.4323% +0.8660%] (p = 0.06 > 0.05)
                        No change in performance detected.
Benchmarking parameterized encode fixed payload/novel-poly-encode/730: Collecting 10 samples in estimated 6.2679 s (110 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/730                        
                        time:   [51.846 ms 51.914 ms 51.962 ms]
                        change: [+2.9079% +3.2654% +3.5350%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode fixed payload/novel-poly-encode/796: Collecting 10 samples in estimated 5.9868 s (165 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/796                        
                        time:   [27.732 ms 27.843 ms 27.957 ms]
                        change: [+3.7564% +4.1554% +4.6107%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized encode fixed payload/novel-poly-encode/862: Collecting 10 samples in estimated 6.5991 s (220 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/862                        
                        time:   [27.414 ms 27.488 ms 27.557 ms]
                        change: [+0.6770% +1.1796% +1.7097%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized encode fixed payload/novel-poly-encode/928: Collecting 10 samples in estimated 5.5208 s (165 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/928                        
                        time:   [27.541 ms 27.812 ms 28.024 ms]
                        change: [+0.5307% +1.2063% +1.8771%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Benchmarking parameterized encode fixed payload/novel-poly-encode/994: Collecting 10 samples in estimated 6.1139 s (220 iterat                                                                                                                              parameterized encode fixed payload/novel-poly-encode/994                        
                        time:   [28.048 ms 28.516 ms 28.915 ms]
                        change: [+0.6676% +1.8381% +3.1353%] (p = 0.00 < 0.05)
                        Change within noise threshold.

Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/4: Collecting 10 samples in estimated 5.0012 s (14                                                                                                                              parameterized reconstruct fixed payload/novel-poly-reconstruct/4                        
                        time:   [33.156 µs 33.355 µs 33.676 µs]
                        change: [-6.2387% -4.6504% -3.1204%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/70: Collecting 10 samples in estimated 8.4051 s (1                                                                                                                              parameterized reconstruct fixed payload/novel-poly-reconstruct/70                        
                        time:   [72.185 ms 72.341 ms 72.698 ms]
                        change: [-18.122% -17.062% -15.934%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/136: Collecting 10 samples in estimated 8.5586 s (                                                                                                                              parameterized reconstruct fixed payload/novel-poly-reconstruct/136                        
                        time:   [77.096 ms 77.372 ms 77.903 ms]
                        change: [-2.0825% -0.3062% +1.6248%] (p = 0.75 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/202: Collecting 10 samples in estimated 6.8974 s (                                                                                                                              parameterized reconstruct fixed payload/novel-poly-reconstruct/202                        
                        time:   [41.785 ms 42.102 ms 42.461 ms]
                        change: [-1.8074% -0.4531% +0.8970%] (p = 0.55 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/268: Collecting 10 samples in estimated 9.8923 s (                                                                                                                              parameterized reconstruct fixed payload/novel-poly-reconstruct/268                        
                        time:   [82.805 ms 83.605 ms 84.788 ms]
                        change: [-0.7433% +0.8343% +2.6662%] (p = 0.34 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/334: Collecting 10 samples in estimated 9.8431 s (                                                                                                                              parameterized reconstruct fixed payload/novel-poly-reconstruct/334                        
                        time:   [88.214 ms 88.455 ms 88.795 ms]
                        change: [-0.6356% +0.9374% +2.5961%] (p = 0.29 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/400: Collecting 10 samples in estimated 5.1090 s (                                                                                                                              parameterized reconstruct fixed payload/novel-poly-reconstruct/400                        
                        time:   [46.138 ms 46.437 ms 46.830 ms]
                        change: [-1.1516% +0.4679% +2.0814%] (p = 0.60 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/466: Collecting 10 samples in estimated 7.4548 s (                                                                                                                              parameterized reconstruct fixed payload/novel-poly-reconstruct/466                        
                        time:   [45.338 ms 45.464 ms 45.607 ms]
                        change: [-5.6881% -3.7581% -1.8554%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/532: Collecting 10 samples in estimated 9.5667 s (                                                                                                                              parameterized reconstruct fixed payload/novel-poly-reconstruct/532                        
                        time:   [87.509 ms 87.867 ms 88.252 ms]
                        change: [-5.6407% -4.7980% -3.8126%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/598: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 5.2s or enable flat sampling.
parameterized reconstruct fixed payload/novel-poly-reconstruct/598                                                                          
                        time:   [93.638 ms 93.805 ms 94.057 ms]
                        change: [+1.0113% +2.0420% +3.0251%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/664: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 5.2s or enable flat sampling.
parameterized reconstruct fixed payload/novel-poly-reconstruct/664                                                                          
                        time:   [94.666 ms 95.346 ms 96.151 ms]
                        change: [+0.1129% +1.9582% +4.1042%] (p = 0.08 > 0.05)
                        No change in performance detected.
Benchmarking parameterized reconstruct fixed payload/novel-poly-reconstruct/730: Warming up for 100.00 ms
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 5.3s or enable flat sampling.
parameterized reconstruct fixed payload/novel-poly-reconstruct/730                                                                          
                        time:   [90.105 ms 90.311 ms 90.722 ms]
                        change: [-4.4338% -2.3209% +0.1337%] (p = 0.09 > 0.05)
                        No change in performance detected.
parameterized reconstruct fixed payload/novel-poly-reconstruct/796                                                                           
                        time:   [50.469 ms 50.917 ms 51.237 ms]
                        change: [+4.4306% +5.3772% +6.4495%] (p = 0.00 < 0.05)
                        Performance has regressed.
parameterized reconstruct fixed payload/novel-poly-reconstruct/862                                                                           
                        time:   [50.287 ms 50.489 ms 50.747 ms]
                        change: [+3.1639% +4.4400% +6.3080%] (p = 0.00 < 0.05)
                        Performance has regressed.
parameterized reconstruct fixed payload/novel-poly-reconstruct/928                                                                           
                        time:   [47.991 ms 48.088 ms 48.180 ms]
                        change: [-3.5466% -3.0325% -2.5615%] (p = 0.00 < 0.05)
                        Performance has improved.
parameterized reconstruct fixed payload/novel-poly-reconstruct/994                                                                           
                        time:   [49.589 ms 49.699 ms 49.799 ms]
                        change: [+1.9317% +2.3540% +2.7583%] (p = 0.00 < 0.05)
                        Performance has regressed.
ordian commented 1 year ago

Have you looked into these severe regressions?

Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/7778000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/7778000                        
                        time:   [2.2722 ms 2.2900 ms 2.3265 ms]
                        change: [+416.77% +430.28% +445.18%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking parameterized reconstruct validator_count=4/novel-poly-reconstruct/8889000: Collecting 10 samples in estimated 5.                                                                                                                              parameterized reconstruct validator_count=4/novel-poly-reconstruct/8889000                        
                        time:   [3.2475 ms 3.2885 ms 3.3559 ms]
                        change: [+31.146% +33.674% +36.508%] (p = 0.00 < 0.05)
drahnr commented 1 year ago

I think that was an artifact of rust analyzer kicking in. I have to rerun them again