cmpute / dashu

A library set of arbitrary precision numbers implemented in Rust.
Apache License 2.0
77 stars 9 forks source link

reimplement `lowest_dword` and `highest_dword` without `unsafe` blocks #40

Open eduardosm opened 1 year ago

cmpute commented 1 year ago

Thanks for the proposal! I'm not sure if this change results in a performance change, I will merge it after I have a better benchmark implementation, and confirmed that this won't result in performance regression.

cmpute commented 10 months ago

This is the change (compared to the master branch) measured on my PC:

Criterion logs ``` ubig_add/10 time: [2.3766 ns 2.3826 ns 2.3928 ns] change: [-1.7571% -0.9668% -0.3367%] (p = 0.00 < 0.05) Change within noise threshold. Found 12 outliers among 100 measurements (12.00%) 5 (5.00%) high mild 7 (7.00%) high severe ubig_add/100 time: [2.3788 ns 2.3814 ns 2.3847 ns] change: [-1.3252% -0.3404% +0.2998%] (p = 0.56 > 0.05) No change in performance detected. Found 8 outliers among 100 measurements (8.00%) 2 (2.00%) high mild 6 (6.00%) high severe ubig_add/1000 time: [50.060 ns 50.089 ns 50.120 ns] change: [+2.2968% +2.4495% +2.5891%] (p = 0.00 < 0.05) Performance has regressed. Found 8 outliers among 100 measurements (8.00%) 2 (2.00%) low severe 1 (1.00%) low mild 2 (2.00%) high mild 3 (3.00%) high severe ubig_add/10000 time: [131.45 ns 131.48 ns 131.50 ns] change: [-1.2042% -0.0828% +0.8478%] (p = 0.87 > 0.05) No change in performance detected. Found 7 outliers among 100 measurements (7.00%) 1 (1.00%) low mild 2 (2.00%) high mild 4 (4.00%) high severe ubig_add/100000 time: [1.0678 µs 1.0679 µs 1.0681 µs] change: [-1.3200% -0.2199% +0.6450%] (p = 0.71 > 0.05) No change in performance detected. Found 9 outliers among 100 measurements (9.00%) 3 (3.00%) low mild 1 (1.00%) high mild 5 (5.00%) high severe ubig_add/1000000 time: [10.746 µs 10.754 µs 10.762 µs] change: [+1.8761% +2.3266% +2.6130%] (p = 0.00 < 0.05) Performance has regressed. Found 6 outliers among 100 measurements (6.00%) 2 (2.00%) low mild 2 (2.00%) high mild 2 (2.00%) high severe ubig_sub/10 time: [2.5913 ns 2.5927 ns 2.5945 ns] change: [-1.2524% -0.4646% +0.0842%] (p = 0.21 > 0.05) No change in performance detected. Found 12 outliers among 100 measurements (12.00%) 7 (7.00%) high mild 5 (5.00%) high severe ubig_sub/100 time: [2.5978 ns 2.6020 ns 2.6084 ns] change: [-0.4112% +0.3007% +1.0557%] (p = 0.52 > 0.05) No change in performance detected. Found 7 outliers among 100 measurements (7.00%) 7 (7.00%) high severe ubig_sub/1000 time: [46.024 ns 46.058 ns 46.095 ns] change: [-0.9272% -0.7540% -0.6216%] (p = 0.00 < 0.05) Change within noise threshold. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high mild ubig_sub/10000 time: [129.48 ns 129.53 ns 129.60 ns] change: [-0.0875% +0.1387% +0.3679%] (p = 0.27 > 0.05) No change in performance detected. Found 11 outliers among 100 measurements (11.00%) 4 (4.00%) high mild 7 (7.00%) high severe ubig_sub/100000 time: [1.0515 µs 1.0518 µs 1.0522 µs] change: [+0.2676% +0.5957% +1.2168%] (p = 0.00 < 0.05) Change within noise threshold. Found 15 outliers among 100 measurements (15.00%) 3 (3.00%) high mild 12 (12.00%) high severe ubig_sub/1000000 time: [10.780 µs 10.787 µs 10.794 µs] change: [-0.2890% +0.1041% +0.5243%] (p = 0.65 > 0.05) No change in performance detected. Found 6 outliers among 100 measurements (6.00%) 3 (3.00%) high mild 3 (3.00%) high severe ubig_mul/10 time: [2.5911 ns 2.5924 ns 2.5942 ns] change: [-0.3656% -0.2453% -0.1328%] (p = 0.00 < 0.05) Change within noise threshold. Found 8 outliers among 100 measurements (8.00%) 5 (5.00%) high mild 3 (3.00%) high severe ubig_mul/100 time: [37.412 ns 37.442 ns 37.473 ns] change: [+0.0746% +0.2835% +0.6538%] (p = 0.03 < 0.05) Change within noise threshold. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe ubig_mul/1000 time: [240.02 ns 240.19 ns 240.36 ns] change: [-8.5700% -8.0916% -7.8104%] (p = 0.00 < 0.05) Performance has improved. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild ubig_mul/10000 time: [10.780 µs 10.787 µs 10.797 µs] change: [-0.5287% -0.4380% -0.3407%] (p = 0.00 < 0.05) Change within noise threshold. Found 10 outliers among 100 measurements (10.00%) 4 (4.00%) high mild 6 (6.00%) high severe ubig_mul/100000 time: [363.46 µs 363.69 µs 363.97 µs] change: [-0.6140% -0.4806% -0.3164%] (p = 0.00 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 3 (3.00%) high mild 4 (4.00%) high severe ubig_mul/1000000 time: [10.948 ms 10.952 ms 10.959 ms] change: [-0.7891% -0.6322% -0.5147%] (p = 0.00 < 0.05) Change within noise threshold. Found 9 outliers among 100 measurements (9.00%) 6 (6.00%) high mild 3 (3.00%) high severe ubig_div/10 time: [3.8966 ns 3.9000 ns 3.9050 ns] change: [-0.6814% +0.1387% +0.9646%] (p = 0.78 > 0.05) No change in performance detected. Found 14 outliers among 100 measurements (14.00%) 10 (10.00%) high mild 4 (4.00%) high severe ubig_div/100 time: [5.8505 ns 5.8535 ns 5.8567 ns] change: [-0.1496% +0.1759% +0.7399%] (p = 0.68 > 0.05) No change in performance detected. Found 2 outliers among 100 measurements (2.00%) 1 (1.00%) high mild 1 (1.00%) high severe ubig_div/1000 time: [138.49 ns 138.59 ns 138.68 ns] change: [-2.0449% -1.6077% -1.3520%] (p = 0.00 < 0.05) Performance has improved. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild ubig_div/10000 time: [549.48 ns 549.79 ns 550.15 ns] change: [-4.2933% -4.2195% -4.1450%] (p = 0.00 < 0.05) Performance has improved. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild ubig_div/100000 time: [4.7185 µs 4.7211 µs 4.7244 µs] change: [-3.6585% -3.1893% -2.9112%] (p = 0.00 < 0.05) Performance has improved. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild ubig_div/1000000 time: [20.232 µs 20.259 µs 20.291 µs] change: [-7.1936% -6.7901% -6.1612%] (p = 0.00 < 0.05) Performance has improved. Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe ubig_gcd/10 time: [19.283 ns 19.293 ns 19.303 ns] change: [+1.2658% +1.3731% +1.5100%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe ubig_gcd/100 time: [131.73 ns 131.78 ns 131.84 ns] change: [-0.9228% -0.3202% +0.4569%] (p = 0.40 > 0.05) No change in performance detected. Found 5 outliers among 100 measurements (5.00%) 2 (2.00%) high mild 3 (3.00%) high severe ubig_gcd/1000 time: [2.0798 µs 2.0812 µs 2.0831 µs] change: [-3.0382% -2.7698% -2.5700%] (p = 0.00 < 0.05) Performance has improved. Found 9 outliers among 100 measurements (9.00%) 3 (3.00%) low mild 2 (2.00%) high mild 4 (4.00%) high severe ubig_gcd/10000 time: [71.134 µs 71.176 µs 71.223 µs] change: [-1.3727% -0.9261% -0.4346%] (p = 0.00 < 0.05) Change within noise threshold. Found 10 outliers among 100 measurements (10.00%) 8 (8.00%) high mild 2 (2.00%) high severe ubig_gcd/100000 time: [3.2690 ms 3.2704 ms 3.2719 ms] change: [+0.6624% +0.7246% +0.7914%] (p = 0.00 < 0.05) Change within noise threshold. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe Benchmarking ubig_gcd/1000000: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 25.6s, or reduce sample count to 10. ubig_gcd/1000000 time: [254.11 ms 254.15 ms 254.19 ms] change: [+0.5439% +0.5918% +0.6350%] (p = 0.00 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 4 (4.00%) low mild 3 (3.00%) high mild ubig_gcd_ext/10 time: [34.934 ns 34.968 ns 35.003 ns] change: [-1.1439% -0.9636% -0.7787%] (p = 0.00 < 0.05) Change within noise threshold. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) low mild 1 (1.00%) high mild ubig_gcd_ext/100 time: [368.67 ns 368.78 ns 368.90 ns] change: [-1.2617% -0.5198% -0.0058%] (p = 0.11 > 0.05) No change in performance detected. Found 9 outliers among 100 measurements (9.00%) 4 (4.00%) high mild 5 (5.00%) high severe ubig_gcd_ext/1000 time: [3.2952 µs 3.2975 µs 3.3000 µs] change: [-1.1628% -0.9376% -0.7853%] (p = 0.00 < 0.05) Change within noise threshold. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild ubig_gcd_ext/10000 time: [150.97 µs 151.04 µs 151.10 µs] change: [+5.0627% +5.1431% +5.2219%] (p = 0.00 < 0.05) Performance has regressed. Found 2 outliers among 100 measurements (2.00%) 1 (1.00%) high mild 1 (1.00%) high severe ubig_gcd_ext/100000 time: [7.5103 ms 7.5135 ms 7.5166 ms] change: [+6.6001% +6.6623% +6.7207%] (p = 0.00 < 0.05) Performance has regressed. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) low mild ubig_to_hex/10 time: [32.585 ns 32.609 ns 32.637 ns] change: [+0.4107% +0.5085% +0.6139%] (p = 0.00 < 0.05) Change within noise threshold. Found 9 outliers among 100 measurements (9.00%) 4 (4.00%) high mild 5 (5.00%) high severe ubig_to_hex/100 time: [64.347 ns 64.402 ns 64.452 ns] change: [+0.3755% +0.6206% +0.7879%] (p = 0.00 < 0.05) Change within noise threshold. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild ubig_to_hex/1000 time: [870.30 ns 893.75 ns 915.33 ns] change: [-7.1609% -4.9631% -2.5903%] (p = 0.00 < 0.05) Performance has improved. Found 21 outliers among 100 measurements (21.00%) 1 (1.00%) high mild 20 (20.00%) high severe ubig_to_hex/10000 time: [8.8814 µs 9.1259 µs 9.3316 µs] change: [-9.0021% -6.2413% -3.4007%] (p = 0.00 < 0.05) Performance has improved. ubig_to_hex/100000 time: [98.191 µs 98.733 µs 99.295 µs] change: [-2.1150% -1.0092% +0.1223%] (p = 0.09 > 0.05) No change in performance detected. Found 8 outliers among 100 measurements (8.00%) 8 (8.00%) low mild ubig_to_hex/1000000 time: [908.07 µs 924.36 µs 937.29 µs] change: [-10.331% -8.1733% -5.8538%] (p = 0.00 < 0.05) Performance has improved. ubig_to_dec/10 time: [35.102 ns 35.126 ns 35.153 ns] change: [-2.2926% -2.0562% -1.8874%] (p = 0.00 < 0.05) Performance has improved. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high mild ubig_to_dec/100 time: [88.054 ns 88.119 ns 88.187 ns] change: [-1.0931% -0.8217% -0.5082%] (p = 0.00 < 0.05) Change within noise threshold. Found 8 outliers among 100 measurements (8.00%) 1 (1.00%) low mild 3 (3.00%) high mild 4 (4.00%) high severe ubig_to_dec/1000 time: [1.4657 µs 1.4664 µs 1.4672 µs] change: [+1.0217% +1.2612% +1.5881%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high severe ubig_to_dec/10000 time: [31.089 µs 31.114 µs 31.136 µs] change: [-3.2214% -3.1253% -3.0342%] (p = 0.00 < 0.05) Performance has improved. Found 10 outliers among 100 measurements (10.00%) 8 (8.00%) high mild 2 (2.00%) high severe ubig_to_dec/100000 time: [901.84 µs 902.26 µs 902.74 µs] change: [-3.3669% -3.1133% -2.9524%] (p = 0.00 < 0.05) Performance has improved. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild ubig_to_dec/1000000 time: [30.629 ms 30.637 ms 30.645 ms] change: [-2.1484% -2.1023% -2.0595%] (p = 0.00 < 0.05) Performance has improved. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild ubig_from_hex/10 time: [7.3670 ns 7.3758 ns 7.3838 ns] change: [-7.9997% -7.5176% -6.5931%] (p = 0.00 < 0.05) Performance has improved. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ubig_from_hex/100 time: [58.681 ns 58.748 ns 58.810 ns] change: [+2.6084% +2.7407% +2.8640%] (p = 0.00 < 0.05) Performance has regressed. Found 9 outliers among 100 measurements (9.00%) 2 (2.00%) low mild 7 (7.00%) high mild ubig_from_hex/1000 time: [288.49 ns 290.53 ns 292.86 ns] change: [+11.788% +12.382% +12.984%] (p = 0.00 < 0.05) Performance has regressed. Found 13 outliers among 100 measurements (13.00%) 5 (5.00%) high mild 8 (8.00%) high severe ubig_from_hex/10000 time: [3.0073 µs 3.0105 µs 3.0140 µs] change: [+15.141% +15.519% +16.075%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe ubig_from_hex/100000 time: [68.638 µs 69.433 µs 70.293 µs] change: [+12.377% +13.644% +15.003%] (p = 0.00 < 0.05) Performance has regressed. ubig_from_hex/1000000 time: [781.84 µs 782.31 µs 782.80 µs] change: [+5.3228% +5.5885% +5.9746%] (p = 0.00 < 0.05) Performance has regressed. Found 7 outliers among 100 measurements (7.00%) 3 (3.00%) low mild 3 (3.00%) high mild 1 (1.00%) high severe ubig_from_dec/10 time: [7.6068 ns 7.6205 ns 7.6346 ns] change: [-6.1400% -5.4545% -4.9174%] (p = 0.00 < 0.05) Performance has improved. Found 9 outliers among 100 measurements (9.00%) 7 (7.00%) high mild 2 (2.00%) high severe ubig_from_dec/100 time: [62.333 ns 62.409 ns 62.491 ns] change: [-7.5952% -7.4169% -7.2264%] (p = 0.00 < 0.05) Performance has improved. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ubig_from_dec/1000 time: [308.64 ns 308.81 ns 309.01 ns] change: [-10.218% -10.148% -10.076%] (p = 0.00 < 0.05) Performance has improved. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high severe ubig_from_dec/10000 time: [10.543 µs 10.561 µs 10.580 µs] change: [-5.7479% -5.5802% -5.4049%] (p = 0.00 < 0.05) Performance has improved. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high mild ubig_from_dec/100000 time: [551.55 µs 551.83 µs 552.13 µs] change: [-1.4908% -1.2469% -0.7224%] (p = 0.00 < 0.05) Change within noise threshold. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe ubig_from_dec/1000000 time: [17.132 ms 17.141 ms 17.151 ms] change: [-0.7778% -0.7152% -0.6462%] (p = 0.00 < 0.05) Change within noise threshold. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high severe ubig_pow/10 time: [14.833 ns 14.836 ns 14.841 ns] change: [-0.5546% -0.3332% +0.0315%] (p = 0.03 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 3 (3.00%) high mild 4 (4.00%) high severe ubig_pow/100 time: [81.988 ns 82.090 ns 82.245 ns] change: [+1.6384% +1.9623% +2.3475%] (p = 0.00 < 0.05) Performance has regressed. Found 13 outliers among 100 measurements (13.00%) 4 (4.00%) low mild 3 (3.00%) high mild 6 (6.00%) high severe ubig_pow/1000 time: [277.35 ns 277.59 ns 277.84 ns] change: [+4.7760% +5.1974% +5.8051%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) low mild 1 (1.00%) high mild 1 (1.00%) high severe ubig_pow/10000 time: [10.874 µs 10.880 µs 10.887 µs] change: [+0.3980% +0.4809% +0.5690%] (p = 0.00 < 0.05) Change within noise threshold. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high severe ubig_pow/100000 time: [391.49 µs 391.58 µs 391.68 µs] change: [-0.6041% -0.5304% -0.4562%] (p = 0.00 < 0.05) Change within noise threshold. Found 4 outliers among 100 measurements (4.00%) 1 (1.00%) high mild 3 (3.00%) high severe ubig_pow/1000000 time: [12.277 ms 12.284 ms 12.292 ms] change: [-0.0600% +0.0114% +0.0844%] (p = 0.76 > 0.05) No change in performance detected. ubig_modulo_mul/10 time: [2.6079 ns 2.6087 ns 2.6095 ns] change: [-0.6779% -0.3301% -0.1282%] (p = 0.01 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 5 (5.00%) high mild 2 (2.00%) high severe ubig_modulo_mul/100 time: [14.213 ns 14.235 ns 14.258 ns] change: [+1.6330% +1.8475% +2.0638%] (p = 0.00 < 0.05) Performance has regressed. Found 12 outliers among 100 measurements (12.00%) 4 (4.00%) low severe 6 (6.00%) low mild 1 (1.00%) high mild 1 (1.00%) high severe ubig_modulo_mul/1000 time: [662.35 ns 662.73 ns 663.16 ns] change: [-4.8787% -4.3582% -4.0590%] (p = 0.00 < 0.05) Performance has improved. ubig_modulo_mul/10000 time: [29.164 µs 29.176 µs 29.188 µs] change: [-2.7224% -2.1030% -1.4202%] (p = 0.00 < 0.05) Performance has improved. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe Benchmarking ubig_modulo_mul/100000: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.9s, enable flat sampling, or reduce sample count to 60. ubig_modulo_mul/100000 time: [1.1353 ms 1.1360 ms 1.1367 ms] change: [-1.4650% -1.3208% -1.1870%] (p = 0.00 < 0.05) Performance has improved. Found 6 outliers among 100 measurements (6.00%) 2 (2.00%) high mild 4 (4.00%) high severe ubig_modulo_mul/1000000 time: [37.341 ms 37.361 ms 37.381 ms] change: [-0.9953% -0.9410% -0.8789%] (p = 0.00 < 0.05) Change within noise threshold. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild ubig_modulo_pow/10 time: [57.620 ns 57.634 ns 57.652 ns] change: [+0.0108% +0.2619% +0.7187%] (p = 0.21 > 0.05) No change in performance detected. Found 10 outliers among 100 measurements (10.00%) 3 (3.00%) high mild 7 (7.00%) high severe ubig_modulo_pow/100 time: [1.9899 µs 1.9906 µs 1.9915 µs] change: [-1.2649% -0.3629% +0.2346%] (p = 0.53 > 0.05) No change in performance detected. Found 6 outliers among 100 measurements (6.00%) 2 (2.00%) high mild 4 (4.00%) high severe ubig_modulo_pow/1000 time: [632.80 µs 633.12 µs 633.50 µs] change: [-2.1024% -1.9849% -1.8862%] (p = 0.00 < 0.05) Performance has improved. Found 10 outliers among 100 measurements (10.00%) 1 (1.00%) low mild 5 (5.00%) high mild 4 (4.00%) high severe ubig_modulo_pow/10000 time: [327.17 ms 327.30 ms 327.50 ms] change: [-2.3743% -2.3072% -2.2375%] (p = 0.00 < 0.05) Performance has improved. Found 2 outliers among 10 measurements (20.00%) 2 (20.00%) high severe ```

In summary main regressions come from ubig_gcd_ext and ubig_from_hex, improvements come from ubig_div and ubig_from_dec. It seems pretty weird to me..

It will be great if you can provide some benchmark results on your computer. (just run cargo bench -p dashu-int --bench primitive --features rand)

eduardosm commented 9 months ago

I rebased on top of latest master and ran the benchmark (first on master and then on my branch):

Criterion logs ``` ubig_add/10 time: [2.2662 ns 2.2785 ns 2.2912 ns] change: [-0.7254% -0.1897% +0.3681%] (p = 0.49 > 0.05) No change in performance detected. Found 8 outliers among 100 measurements (8.00%) 1 (1.00%) low mild 3 (3.00%) high mild 4 (4.00%) high severe ubig_add/100 time: [2.2624 ns 2.2706 ns 2.2789 ns] change: [-0.4885% +0.0759% +0.6043%] (p = 0.80 > 0.05) No change in performance detected. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ubig_add/1000 time: [26.960 ns 27.066 ns 27.174 ns] change: [-6.1862% -5.6952% -5.2172%] (p = 0.00 < 0.05) Performance has improved. Found 18 outliers among 100 measurements (18.00%) 11 (11.00%) high mild 7 (7.00%) high severe ubig_add/10000 time: [153.91 ns 154.54 ns 155.15 ns] change: [-4.6586% -4.1972% -3.7001%] (p = 0.00 < 0.05) Performance has improved. Found 13 outliers among 100 measurements (13.00%) 9 (9.00%) high mild 4 (4.00%) high severe ubig_add/100000 time: [1.2051 µs 1.2092 µs 1.2132 µs] change: [+0.5259% +1.1385% +1.7197%] (p = 0.00 < 0.05) Change within noise threshold. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild ubig_add/1000000 time: [13.089 µs 13.131 µs 13.171 µs] change: [+1.3662% +1.8442% +2.2806%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 4 (4.00%) high mild 1 (1.00%) high severe ubig_sub/10 time: [2.1783 ns 2.1866 ns 2.1953 ns] change: [-0.7464% -0.2410% +0.3201%] (p = 0.38 > 0.05) No change in performance detected. Found 4 outliers among 100 measurements (4.00%) 2 (2.00%) high mild 2 (2.00%) high severe ubig_sub/100 time: [2.1663 ns 2.1754 ns 2.1842 ns] change: [-0.6212% +0.1472% +0.9848%] (p = 0.72 > 0.05) No change in performance detected. Found 7 outliers among 100 measurements (7.00%) 1 (1.00%) low mild 2 (2.00%) high mild 4 (4.00%) high severe ubig_sub/1000 time: [24.239 ns 24.329 ns 24.423 ns] change: [-9.9070% -9.4833% -9.0303%] (p = 0.00 < 0.05) Performance has improved. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ubig_sub/10000 time: [154.59 ns 155.17 ns 155.72 ns] change: [-0.2034% +0.2332% +0.6603%] (p = 0.29 > 0.05) No change in performance detected. Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe ubig_sub/100000 time: [1.2030 µs 1.2071 µs 1.2112 µs] change: [-0.1544% +0.3360% +0.8195%] (p = 0.18 > 0.05) No change in performance detected. Found 19 outliers among 100 measurements (19.00%) 7 (7.00%) high mild 12 (12.00%) high severe ubig_sub/1000000 time: [12.833 µs 12.884 µs 12.938 µs] change: [-3.9248% -3.4457% -2.9751%] (p = 0.00 < 0.05) Performance has improved. Found 6 outliers among 100 measurements (6.00%) 3 (3.00%) high mild 3 (3.00%) high severe ubig_mul/10 time: [2.3782 ns 2.3895 ns 2.4001 ns] change: [-0.0838% +0.5038% +1.0713%] (p = 0.09 > 0.05) No change in performance detected. Found 17 outliers among 100 measurements (17.00%) 8 (8.00%) high mild 9 (9.00%) high severe ubig_mul/100 time: [18.029 ns 18.094 ns 18.159 ns] change: [-11.259% -10.792% -10.335%] (p = 0.00 < 0.05) Performance has improved. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high severe ubig_mul/1000 time: [201.19 ns 202.02 ns 202.85 ns] change: [-5.4438% -4.8514% -4.1836%] (p = 0.00 < 0.05) Performance has improved. Found 6 outliers among 100 measurements (6.00%) 2 (2.00%) high mild 4 (4.00%) high severe ubig_mul/10000 time: [10.725 µs 10.773 µs 10.821 µs] change: [+0.9219% +1.5121% +2.0821%] (p = 0.00 < 0.05) Change within noise threshold. Found 2 outliers among 100 measurements (2.00%) 1 (1.00%) high mild 1 (1.00%) high severe ubig_mul/100000 time: [373.85 µs 374.80 µs 375.75 µs] change: [+2.4541% +2.8445% +3.2626%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe ubig_mul/1000000 time: [11.557 ms 11.591 ms 11.627 ms] change: [+2.9965% +3.3795% +3.7682%] (p = 0.00 < 0.05) Performance has regressed. ubig_div/10 time: [8.5139 ns 8.5442 ns 8.5740 ns] change: [-2.1899% -1.7101% -1.1981%] (p = 0.00 < 0.05) Performance has improved. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ubig_div/100 time: [22.175 ns 22.252 ns 22.328 ns] change: [+0.3468% +0.7901% +1.2527%] (p = 0.00 < 0.05) Change within noise threshold. Found 4 outliers among 100 measurements (4.00%) 2 (2.00%) high mild 2 (2.00%) high severe ubig_div/1000 time: [116.89 ns 117.41 ns 117.93 ns] change: [+1.0087% +1.5553% +2.1211%] (p = 0.00 < 0.05) Performance has regressed. Found 6 outliers among 100 measurements (6.00%) 5 (5.00%) high mild 1 (1.00%) high severe ubig_div/10000 time: [662.61 ns 665.16 ns 667.67 ns] change: [+5.1698% +5.6281% +6.0460%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high mild ubig_div/100000 time: [5.3078 µs 5.3262 µs 5.3442 µs] change: [+0.4762% +0.9707% +1.4603%] (p = 0.00 < 0.05) Change within noise threshold. Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe ubig_div/1000000 time: [21.611 µs 21.679 µs 21.747 µs] change: [-3.6796% -3.2114% -2.6330%] (p = 0.00 < 0.05) Performance has improved. Found 5 outliers among 100 measurements (5.00%) 4 (4.00%) high mild 1 (1.00%) high severe ubig_gcd/10 time: [18.153 ns 18.217 ns 18.279 ns] change: [+0.1918% +0.9254% +1.6031%] (p = 0.01 < 0.05) Change within noise threshold. Found 9 outliers among 100 measurements (9.00%) 5 (5.00%) high mild 4 (4.00%) high severe ubig_gcd/100 time: [161.11 ns 161.63 ns 162.14 ns] change: [-0.6235% -0.0725% +0.4367%] (p = 0.80 > 0.05) No change in performance detected. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high severe ubig_gcd/1000 time: [5.1609 µs 5.1840 µs 5.2075 µs] change: [-1.2559% -0.4832% +0.1890%] (p = 0.20 > 0.05) No change in performance detected. Found 6 outliers among 100 measurements (6.00%) 4 (4.00%) high mild 2 (2.00%) high severe ubig_gcd/10000 time: [100.87 µs 101.33 µs 101.74 µs] change: [-0.4313% +0.1016% +0.6889%] (p = 0.72 > 0.05) No change in performance detected. Found 4 outliers among 100 measurements (4.00%) 2 (2.00%) high mild 2 (2.00%) high severe ubig_gcd/100000 time: [3.7758 ms 3.7844 ms 3.7934 ms] change: [-0.2450% +0.0760% +0.4147%] (p = 0.66 > 0.05) No change in performance detected. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild Benchmarking ubig_gcd/1000000: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 24.6s, or reduce sample count to 20. ubig_gcd/1000000 time: [243.60 ms 243.89 ms 244.21 ms] change: [-0.2347% -0.0715% +0.1058%] (p = 0.43 > 0.05) No change in performance detected. Found 11 outliers among 100 measurements (11.00%) 5 (5.00%) high mild 6 (6.00%) high severe ubig_gcd_ext/10 time: [50.010 ns 50.094 ns 50.197 ns] change: [+1.0297% +1.5403% +2.0826%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe ubig_gcd_ext/100 time: [839.45 ns 841.91 ns 844.75 ns] change: [-0.1932% +0.3139% +0.8462%] (p = 0.22 > 0.05) No change in performance detected. Found 2 outliers among 100 measurements (2.00%) 1 (1.00%) high mild 1 (1.00%) high severe ubig_gcd_ext/1000 time: [6.7498 µs 6.7672 µs 6.7850 µs] change: [-1.0595% -0.5498% -0.0749%] (p = 0.04 < 0.05) Change within noise threshold. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild ubig_gcd_ext/10000 time: [182.82 µs 183.12 µs 183.47 µs] change: [-0.4809% +0.0492% +0.6107%] (p = 0.86 > 0.05) No change in performance detected. Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe ubig_gcd_ext/100000 time: [8.1848 ms 8.2044 ms 8.2249 ms] change: [-1.3531% -0.9996% -0.6439%] (p = 0.00 < 0.05) Change within noise threshold. ubig_to_hex/10 time: [34.147 ns 34.290 ns 34.435 ns] change: [-0.0113% +0.5168% +1.0301%] (p = 0.05 > 0.05) No change in performance detected. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high mild ubig_to_hex/100 time: [67.037 ns 67.245 ns 67.449 ns] change: [-0.0124% +0.4634% +0.9324%] (p = 0.06 > 0.05) No change in performance detected. Found 7 outliers among 100 measurements (7.00%) 4 (4.00%) high mild 3 (3.00%) high severe ubig_to_hex/1000 time: [825.07 ns 828.55 ns 832.41 ns] change: [-3.7146% -3.1330% -2.5211%] (p = 0.00 < 0.05) Performance has improved. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe ubig_to_hex/10000 time: [7.7207 µs 7.7551 µs 7.7872 µs] change: [-8.5855% -8.0495% -7.5148%] (p = 0.00 < 0.05) Performance has improved. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe ubig_to_hex/100000 time: [77.907 µs 78.166 µs 78.429 µs] change: [-7.5532% -7.0041% -6.4588%] (p = 0.00 < 0.05) Performance has improved. Found 4 outliers among 100 measurements (4.00%) 2 (2.00%) high mild 2 (2.00%) high severe ubig_to_hex/1000000 time: [781.13 µs 782.78 µs 784.67 µs] change: [+0.3349% +0.8254% +1.2662%] (p = 0.00 < 0.05) Change within noise threshold. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild ubig_to_dec/10 time: [36.406 ns 36.521 ns 36.638 ns] change: [+0.2983% +0.8737% +1.6553%] (p = 0.00 < 0.05) Change within noise threshold. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe ubig_to_dec/100 time: [89.965 ns 90.144 ns 90.354 ns] change: [-0.1096% +0.2495% +0.5907%] (p = 0.17 > 0.05) No change in performance detected. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild ubig_to_dec/1000 time: [1.2901 µs 1.2950 µs 1.2998 µs] change: [-0.3612% +0.1547% +0.6607%] (p = 0.57 > 0.05) No change in performance detected. Found 2 outliers among 100 measurements (2.00%) 1 (1.00%) high mild 1 (1.00%) high severe ubig_to_dec/10000 time: [30.277 µs 30.399 µs 30.530 µs] change: [+0.8688% +1.4450% +2.0587%] (p = 0.00 < 0.05) Change within noise threshold. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe ubig_to_dec/100000 time: [923.42 µs 926.22 µs 929.30 µs] change: [+1.8963% +2.3712% +2.8356%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe ubig_to_dec/1000000 time: [31.472 ms 31.574 ms 31.679 ms] change: [+0.8428% +1.2423% +1.6108%] (p = 0.00 < 0.05) Change within noise threshold. Found 11 outliers among 100 measurements (11.00%) 5 (5.00%) low mild 6 (6.00%) high mild ubig_from_hex/10 time: [8.5293 ns 8.5606 ns 8.5904 ns] change: [+0.4383% +1.0903% +1.8122%] (p = 0.00 < 0.05) Change within noise threshold. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high severe ubig_from_hex/100 time: [45.457 ns 45.709 ns 45.961 ns] change: [+1.4911% +1.9664% +2.4542%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe ubig_from_hex/1000 time: [288.77 ns 289.92 ns 291.02 ns] change: [-0.1814% +0.2980% +0.8387%] (p = 0.25 > 0.05) No change in performance detected. Found 15 outliers among 100 measurements (15.00%) 12 (12.00%) high mild 3 (3.00%) high severe ubig_from_hex/10000 time: [4.4116 µs 4.4279 µs 4.4435 µs] change: [+6.8781% +7.4551% +7.9896%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe ubig_from_hex/100000 time: [79.622 µs 79.995 µs 80.359 µs] change: [-3.0620% -2.1824% -1.4488%] (p = 0.00 < 0.05) Performance has improved. Found 10 outliers among 100 measurements (10.00%) 5 (5.00%) low mild 2 (2.00%) high mild 3 (3.00%) high severe ubig_from_hex/1000000 time: [804.84 µs 805.78 µs 806.94 µs] change: [-2.7384% -2.3056% -1.8292%] (p = 0.00 < 0.05) Performance has improved. Found 2 outliers among 100 measurements (2.00%) 1 (1.00%) high mild 1 (1.00%) high severe ubig_from_dec/10 time: [8.8910 ns 8.9203 ns 8.9535 ns] change: [+6.4770% +7.1909% +8.0256%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 1 (1.00%) high mild 4 (4.00%) high severe ubig_from_dec/100 time: [46.718 ns 46.933 ns 47.198 ns] change: [-3.4591% -2.9062% -2.3106%] (p = 0.00 < 0.05) Performance has improved. Found 5 outliers among 100 measurements (5.00%) 4 (4.00%) high mild 1 (1.00%) high severe ubig_from_dec/1000 time: [335.13 ns 336.19 ns 337.25 ns] change: [+3.5038% +3.9987% +4.4845%] (p = 0.00 < 0.05) Performance has regressed. Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe ubig_from_dec/10000 time: [10.410 µs 10.446 µs 10.481 µs] change: [-1.1674% -0.4951% +0.2137%] (p = 0.17 > 0.05) No change in performance detected. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe ubig_from_dec/100000 time: [563.67 µs 565.95 µs 568.27 µs] change: [+1.2907% +1.7581% +2.2250%] (p = 0.00 < 0.05) Performance has regressed. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild ubig_from_dec/1000000 time: [17.595 ms 17.642 ms 17.690 ms] change: [+1.1183% +1.5072% +1.9004%] (p = 0.00 < 0.05) Performance has regressed. ubig_pow/10 time: [18.290 ns 18.362 ns 18.432 ns] change: [-3.0700% -2.5327% -1.9784%] (p = 0.00 < 0.05) Performance has improved. Found 13 outliers among 100 measurements (13.00%) 10 (10.00%) high mild 3 (3.00%) high severe ubig_pow/100 time: [45.333 ns 45.487 ns 45.643 ns] change: [-41.228% -40.894% -40.586%] (p = 0.00 < 0.05) Performance has improved. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ubig_pow/1000 time: [223.38 ns 224.16 ns 224.94 ns] change: [-12.456% -12.057% -11.641%] (p = 0.00 < 0.05) Performance has improved. Found 5 outliers among 100 measurements (5.00%) 4 (4.00%) high mild 1 (1.00%) high severe ubig_pow/10000 time: [10.915 µs 10.961 µs 11.005 µs] change: [+0.0105% +0.4850% +0.9462%] (p = 0.05 < 0.05) Change within noise threshold. Found 9 outliers among 100 measurements (9.00%) 2 (2.00%) low mild 3 (3.00%) high mild 4 (4.00%) high severe ubig_pow/100000 time: [414.26 µs 415.69 µs 417.22 µs] change: [+3.6244% +4.0782% +4.4829%] (p = 0.00 < 0.05) Performance has regressed. Found 4 outliers among 100 measurements (4.00%) 4 (4.00%) high mild ubig_pow/1000000 time: [12.810 ms 12.845 ms 12.882 ms] change: [+2.0501% +2.4824% +2.9337%] (p = 0.00 < 0.05) Performance has regressed. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild ubig_modulo_mul/10 time: [2.7252 ns 2.7349 ns 2.7446 ns] change: [-0.8493% -0.3775% +0.1452%] (p = 0.14 > 0.05) No change in performance detected. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ubig_modulo_mul/100 time: [11.907 ns 11.940 ns 11.974 ns] change: [-0.4121% +0.0230% +0.4731%] (p = 0.91 > 0.05) No change in performance detected. Found 4 outliers among 100 measurements (4.00%) 2 (2.00%) high mild 2 (2.00%) high severe ubig_modulo_mul/1000 time: [589.50 ns 591.59 ns 593.55 ns] change: [+0.5053% +1.0400% +1.5264%] (p = 0.00 < 0.05) Change within noise threshold. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe ubig_modulo_mul/10000 time: [29.198 µs 29.311 µs 29.427 µs] change: [+1.4657% +1.9439% +2.4242%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe Benchmarking ubig_modulo_mul/100000: Warming up for 3.0000 s Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.0s, enable flat sampling, or reduce sample count to 60. ubig_modulo_mul/100000 time: [1.1723 ms 1.1776 ms 1.1828 ms] change: [+2.2390% +2.9308% +3.5952%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 1 (1.00%) high mild 4 (4.00%) high severe ubig_modulo_mul/1000000 time: [38.444 ms 38.600 ms 38.759 ms] change: [+0.5476% +1.0740% +1.5784%] (p = 0.00 < 0.05) Change within noise threshold. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild ubig_modulo_pow/10 time: [61.708 ns 61.819 ns 61.955 ns] change: [-0.2115% +0.2595% +0.6663%] (p = 0.23 > 0.05) No change in performance detected. ubig_modulo_pow/100 time: [2.2382 µs 2.2457 µs 2.2529 µs] change: [-1.3444% -0.8053% -0.1739%] (p = 0.00 < 0.05) Change within noise threshold. Found 19 outliers among 100 measurements (19.00%) 15 (15.00%) high mild 4 (4.00%) high severe ubig_modulo_pow/1000 time: [605.68 µs 607.42 µs 609.33 µs] change: [-0.9285% -0.4388% +0.0211%] (p = 0.07 > 0.05) No change in performance detected. Found 15 outliers among 100 measurements (15.00%) 11 (11.00%) high mild 4 (4.00%) high severe ubig_modulo_pow/10000 time: [331.99 ms 334.23 ms 336.51 ms] change: [+2.2712% +3.1484% +3.9725%] (p = 0.00 < 0.05) Performance has regressed. Found 2 outliers among 10 measurements (20.00%) 1 (10.00%) low mild 1 (10.00%) high mild ```

-40% in ubig_pow/100? I ran it again

ubig_pow/100 again ``` ubig_pow/100 time: [46.602 ns 46.676 ns 46.762 ns] change: [+0.2617% +0.8519% +1.4464%] (p = 0.00 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 4 (4.00%) high mild 3 (3.00%) high severe ```

So it looks like it was a huge outlier as a whole...

cmpute commented 8 months ago

It seems that the benchmark needs improving... I will try this PR before the next major version release, when I will compare again the performance of dashu against other libraries.