shuoli84
commented
1 year ago cursor related bench improved by over 25%. With non-obvious degrade.
Closed shuoli84 closed 1 year ago
shuoli84
commented
1 year ago cursor related bench improved by over 25%. With non-obvious degrade.
clone/bptree/100000 time: [1.0259 ms 1.0513 ms 1.0761 ms] change: [-1.2505% +0.8137% +3.0219%] (p = 0.46 > 0.05) No change in performance detected. Found 11 outliers among 100 measurements (11.00%) 2 (2.00%) low severe 5 (5.00%) high mild 4 (4.00%) high severe
drop/bptree/1000 time: [1.4200 µs 1.4291 µs 1.4391 µs] change: [+3.1978% +3.7508% +4.3447%] (p = 0.00 < 0.05) Performance has regressed. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild Benchmarking drop/bptree/10000: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.6s, enable flat sampling, or reduce sample count to 50. drop/bptree/10000 time: [15.917 µs 15.989 µs 16.061 µs] change: [+1.7962% +2.2403% +2.7293%] (p = 0.00 < 0.05) Performance has regressed. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild drop/bptree/100000 time: [346.44 µs 353.77 µs 360.29 µs] change: [+1.8993% +4.4155% +6.5367%] (p = 0.00 < 0.05) Performance has regressed. Found 15 outliers among 100 measurements (15.00%) 4 (4.00%) low severe 4 (4.00%) low mild 5 (5.00%) high mild 2 (2.00%) high severe
iter/bptree/1000 time: [377.81 ns 378.01 ns 378.24 ns] change: [-10.219% -10.021% -9.8320%] (p = 0.00 < 0.05) Performance has improved. Found 7 outliers among 100 measurements (7.00%) 1 (1.00%) high mild 6 (6.00%) high severe iter/bptree/10000 time: [5.2231 µs 5.2336 µs 5.2453 µs] change: [-3.2394% -3.0327% -2.7999%] (p = 0.00 < 0.05) Performance has improved. Found 7 outliers among 100 measurements (7.00%) 4 (4.00%) high mild 3 (3.00%) high severe iter/bptree/100000 time: [59.010 µs 59.085 µs 59.159 µs] change: [+2.3637% +2.5270% +2.6914%] (p = 0.00 < 0.05) Performance has regressed. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe
into_iter/bptree/1000 time: [1.4044 µs 1.4116 µs 1.4191 µs] change: [+3.1694% +3.8626% +4.6202%] (p = 0.00 < 0.05) Performance has regressed. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe Benchmarking into_iter/bptree/10000: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.6s, enable flat sampling, or reduce sample count to 50. into_iter/bptree/10000 time: [15.406 µs 15.459 µs 15.520 µs] change: [+2.0111% +2.4144% +2.7866%] (p = 0.00 < 0.05) Performance has regressed. into_iter/bptree/100000 time: [333.80 µs 336.99 µs 340.20 µs] change: [+4.6921% +6.2690% +7.7068%] (p = 0.00 < 0.05) Performance has regressed. Found 11 outliers among 100 measurements (11.00%) 2 (2.00%) low severe 4 (4.00%) low mild 2 (2.00%) high mild 3 (3.00%) high severe
ordered_insert/bptree/1000 time: [15.324 µs 15.355 µs 15.383 µs] change: [-0.2714% -0.0046% +0.2609%] (p = 0.98 > 0.05) No change in performance detected. ordered_insert/bptree bulk/1000 time: [6.1010 µs 6.1080 µs 6.1164 µs] change: [-0.1855% +0.0266% +0.2015%] (p = 0.80 > 0.05) No change in performance detected. Found 11 outliers among 100 measurements (11.00%) 2 (2.00%) high mild 9 (9.00%) high severe ordered_insert/bptree/10000 time: [162.61 µs 163.12 µs 163.66 µs] change: [+0.0146% +0.3310% +0.7069%] (p = 0.05 > 0.05) No change in performance detected. Found 6 outliers among 100 measurements (6.00%) 5 (5.00%) high mild 1 (1.00%) high severe ordered_insert/bptree bulk/10000 time: [68.670 µs 68.832 µs 68.994 µs] change: [+0.1456% +0.5004% +0.8521%] (p = 0.01 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 5 (5.00%) low mild 2 (2.00%) high mild ordered_insert/bptree/100000 time: [2.0153 ms 2.0211 ms 2.0276 ms] change: [-0.2155% +0.1531% +0.5516%] (p = 0.44 > 0.05) No change in performance detected. Found 9 outliers among 100 measurements (9.00%) 6 (6.00%) high mild 3 (3.00%) high severe Benchmarking ordered_insert/bptree bulk/100000: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.2s, enable flat sampling, or reduce sample count to 60. ordered_insert/bptree bulk/100000 time: [944.80 µs 953.88 µs 963.74 µs] change: [+1.1987% +2.9603% +4.8597%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe
random_insert/bptree/1000 time: [64.239 µs 64.727 µs 65.278 µs] change: [+5.0642% +5.9125% +6.7275%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 1 (1.00%) low mild 4 (4.00%) high mild random_insert/bptree sort_load/1000 time: [43.186 µs 43.230 µs 43.282 µs] change: [-0.2593% -0.0322% +0.1942%] (p = 0.78 > 0.05) No change in performance detected. Found 7 outliers among 100 measurements (7.00%) 3 (3.00%) high mild 4 (4.00%) high severe Benchmarking random_insert/bptree/10000: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.9s, enable flat sampling, or reduce sample count to 60. random_insert/bptree/10000 time: [1.3664 ms 1.3729 ms 1.3804 ms] change: [-1.9374% -1.5681% -1.1383%] (p = 0.00 < 0.05) Performance has improved. Found 16 outliers among 100 measurements (16.00%) 1 (1.00%) high mild 15 (15.00%) high severe random_insert/bptree sort_load/10000 time: [682.45 µs 684.43 µs 686.70 µs] change: [-0.2468% +0.2891% +0.8435%] (p = 0.29 > 0.05) No change in performance detected. Found 6 outliers among 100 measurements (6.00%) 3 (3.00%) high mild 3 (3.00%) high severe random_insert/bptree/100000 time: [18.257 ms 18.295 ms 18.341 ms] change: [-2.7583% -2.4703% -2.1724%] (p = 0.00 < 0.05) Performance has improved. Found 11 outliers among 100 measurements (11.00%) 4 (4.00%) high mild 7 (7.00%) high severe random_insert/bptree sort_load/100000 time: [9.5430 ms 9.5709 ms 9.6020 ms] change: [+0.1206% +0.4289% +0.7495%] (p = 0.01 < 0.05) Change within noise threshold. Found 16 outliers among 100 measurements (16.00%) 4 (4.00%) high mild 12 (12.00%) high severe
ordered_remove/bptree/1000 time: [61.234 µs 61.438 µs 61.697 µs] change: [-2.3212% -2.0545% -1.7394%] (p = 0.00 < 0.05) Performance has improved. Found 8 outliers among 100 measurements (8.00%) 6 (6.00%) high mild 2 (2.00%) high severe ordered_remove/bptree/10000 time: [635.15 µs 635.52 µs 635.93 µs] change: [-3.5872% -3.4864% -3.3868%] (p = 0.00 < 0.05) Performance has improved. Found 4 outliers among 100 measurements (4.00%) 4 (4.00%) high severe ordered_remove/bptree/100000 time: [7.0737 ms 7.0915 ms 7.1089 ms] change: [-3.1205% -2.6132% -2.1874%] (p = 0.00 < 0.05) Performance has improved. Found 16 outliers among 100 measurements (16.00%) 5 (5.00%) low severe 6 (6.00%) low mild 1 (1.00%) high mild 4 (4.00%) high severe
random_remove/bptree/1000 time: [65.833 µs 66.524 µs 67.329 µs] change: [-7.0403% -6.2420% -5.3182%] (p = 0.00 < 0.05) Performance has improved. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild random_remove/bptree/10000 time: [1.2374 ms 1.2385 ms 1.2398 ms] change: [-4.7078% -4.1740% -3.6455%] (p = 0.00 < 0.05) Performance has improved. Found 15 outliers among 100 measurements (15.00%) 13 (13.00%) high mild 2 (2.00%) high severe random_remove/bptree/100000 time: [16.729 ms 16.760 ms 16.795 ms] change: [-6.0180% -5.0644% -4.1877%] (p = 0.00 < 0.05) Performance has improved. Found 8 outliers among 100 measurements (8.00%) 1 (1.00%) high mild 7 (7.00%) high severe
ordered_get/bptree/1000 time: [10.323 µs 10.341 µs 10.362 µs] change: [-0.4944% -0.1736% +0.1428%] (p = 0.31 > 0.05) No change in performance detected. Found 13 outliers among 100 measurements (13.00%) 7 (7.00%) high mild 6 (6.00%) high severe ordered_get/bptree/10000 time: [111.14 µs 111.48 µs 111.89 µs] change: [-2.5433% -1.9530% -1.4196%] (p = 0.00 < 0.05) Performance has improved. Found 11 outliers among 100 measurements (11.00%) 4 (4.00%) high mild 7 (7.00%) high severe Benchmarking ordered_get/bptree/100000: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.9s, enable flat sampling, or reduce sample count to 60. ordered_get/bptree/100000 time: [1.1562 ms 1.1574 ms 1.1590 ms] change: [-3.8315% -3.2516% -2.6834%] (p = 0.00 < 0.05) Performance has improved. Found 16 outliers among 100 measurements (16.00%) 1 (1.00%) low mild 3 (3.00%) high mild 12 (12.00%) high severe
random_get/bptree/1000 time: [24.061 µs 24.712 µs 25.388 µs] change: [-18.055% -16.000% -13.928%] (p = 0.00 < 0.05) Performance has improved. Found 6 outliers among 100 measurements (6.00%) 2 (2.00%) low mild 3 (3.00%) high mild 1 (1.00%) high severe random_get/bptree/10000 time: [954.35 µs 955.27 µs 956.27 µs] change: [-3.0240% -2.7766% -2.5366%] (p = 0.00 < 0.05) Performance has improved. Found 12 outliers among 100 measurements (12.00%) 1 (1.00%) low mild 7 (7.00%) high mild 4 (4.00%) high severe random_get/bptree/100000 time: [13.671 ms 13.759 ms 13.850 ms] change: [-0.7662% -0.0047% +0.6461%] (p = 0.99 > 0.05) No change in performance detected.
cursor/bptree/1000 time: [3.3011 µs 3.3628 µs 3.4419 µs] change: [-28.066% -26.354% -24.237%] (p = 0.00 < 0.05) Performance has improved. cursor/bptree/10000 time: [35.208 µs 35.280 µs 35.352 µs] change: [-27.750% -27.582% -27.390%] (p = 0.00 < 0.05) Performance has improved. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild cursor/bptree/100000 time: [364.70 µs 365.78 µs 366.96 µs] change: [-25.437% -25.261% -25.085%] (p = 0.00 < 0.05) Performance has improved. Found 1 outliers among 100 measurements (1.00%)