minio / simdjson-go

Golang port of simdjson: parsing gigabytes of JSON per second
Apache License 2.0
1.8k stars 85 forks source link

Remove parsing allocs #55

Closed klauspost closed 2 years ago

klauspost commented 2 years ago

Cleanup: Don't send params already in internalParsedJson.

benchmark                                      old ns/op     new ns/op     delta
BenchmarkParseSmall/copy-32                    1097          758           -30.88%
BenchmarkParseSmall/nocopy-32                  1032          674           -34.64%
BenchmarkParseSmall/nocopy-par-32              103           41.6          -59.44%
BenchmarkParseMedium/copy-32                   4518          4170          -7.70%
BenchmarkParseMedium/nocopy-32                 3830          3251          -15.12%
BenchmarkParseMedium/nocopy-par-32             242           178           -26.35%
BenchmarkParseLarge/copy-32                    57364         58955         +2.77%
BenchmarkParseLarge/nocopy-32                  46456         46993         +1.16%
BenchmarkParseLarge/nocopy-par-32              2569          2525          -1.71%
BenchmarkParseApache_builds/copy-32            136834        131745        -3.72%
BenchmarkParseApache_builds/nocopy-32          101684        95344         -6.24%
BenchmarkParseApache_builds/nocopy-par-32      6981          6895          -1.23%
BenchmarkParseCanada/copy-32                   11885695      12142371      +2.16%
BenchmarkParseCanada/nocopy-32                 11793277      12134212      +2.89%
BenchmarkParseCanada/nocopy-par-32             686432        641823        -6.50%
BenchmarkParseCitm_catalog/copy-32             1685947       1503883       -10.80%
BenchmarkParseCitm_catalog/nocopy-32           1250969       1311906       +4.87%
BenchmarkParseCitm_catalog/nocopy-par-32       89255         91381         +2.38%
BenchmarkParseGithub_events/copy-32            69842         72738         +4.15%
BenchmarkParseGithub_events/nocopy-32          56705         56879         +0.31%
BenchmarkParseGithub_events/nocopy-par-32      3221          3166          -1.71%
BenchmarkParseGsoc_2018/copy-32                1287979       1277811       -0.79%
BenchmarkParseGsoc_2018/nocopy-32              1015391       1014019       -0.14%
BenchmarkParseGsoc_2018/nocopy-par-32          138918        140691        +1.28%
BenchmarkParseInstruments/copy-32              308224        303929        -1.39%
BenchmarkParseInstruments/nocopy-32            264055        259507        -1.72%
BenchmarkParseInstruments/nocopy-par-32        16877         16861         -0.09%
BenchmarkParseMarine_ik/copy-32                13165667      13007582      -1.20%
BenchmarkParseMarine_ik/nocopy-32              12684994      12639697      -0.36%
BenchmarkParseMarine_ik/nocopy-par-32          690559        655854        -5.03%
BenchmarkParseMesh/copy-32                     3833472       3763927       -1.81%
BenchmarkParseMesh/nocopy-32                   3775770       3750355       -0.67%
BenchmarkParseMesh/nocopy-par-32               204998        195802        -4.49%
BenchmarkParseMesh_pretty/copy-32              4251213       4385560       +3.16%
BenchmarkParseMesh_pretty/nocopy-32            4230253       4380091       +3.54%
BenchmarkParseMesh_pretty/nocopy-par-32        240712        237028        -1.53%
BenchmarkParseNumbers/copy-32                  731972        707634        -3.32%
BenchmarkParseNumbers/nocopy-32                732324        706581        -3.52%
BenchmarkParseNumbers/nocopy-par-32            41009         39899         -2.71%
BenchmarkParseRandom/copy-32                   1058452       1066158       +0.73%
BenchmarkParseRandom/nocopy-32                 730790        755489        +3.38%
BenchmarkParseRandom/nocopy-par-32             47112         47751         +1.36%
BenchmarkParseTwitter/copy-32                  629000        631544        +0.40%
BenchmarkParseTwitter/nocopy-32                470858        468639        -0.47%
BenchmarkParseTwitter/nocopy-par-32            34359         34139         -0.64%
BenchmarkParseTwitterEscaped/copy-32           1003597       1006342       +0.27%
BenchmarkParseTwitterEscaped/nocopy-32         869137        866733        -0.28%
BenchmarkParseTwitterEscaped/nocopy-par-32     48923         48474         -0.92%
BenchmarkParseUpdate_center/copy-32            742999        762844        +2.67%
BenchmarkParseUpdate_center/nocopy-32          511315        523626        +2.41%
BenchmarkParseUpdate_center/nocopy-par-32      36312         36520         +0.57%
BenchmarkJsonParserLarge/nocopy-32             54605         58703         +7.50%
BenchmarkParseNumber/Pos/63bit-32              82.1          85.2          +3.68%
BenchmarkParseNumber/Neg/63bit-32              84.2          86.5          +2.84%
BenchmarkParseNumberFloat-32                   62.5          65.3          +4.44%
BenchmarkParseAtof64FloatGolang-32             32.2          32.4          +0.50%
BenchmarkParseNumberFloatExp-32                70.7          81.1          +14.64%
BenchmarkParseNumberBig-32                     132           152           +15.20%
BenchmarkParseNumberRandomBits-32              152           148           -3.02%
BenchmarkParseNumberRandomFloats-32            122           132           +8.89%
BenchmarkParseIntGolang/Pos/63bit-32           29.8          30.0          +0.57%
BenchmarkParseIntGolang/Neg/63bit-32           29.8          29.9          +0.44%

benchmark                                      old MB/s     new MB/s     speedup
BenchmarkParseSmall/copy-32                    172.36       249.27       1.45x
BenchmarkParseSmall/nocopy-32                  183.06       280.19       1.53x
BenchmarkParseSmall/nocopy-par-32              1840.85      4537.99      2.47x
BenchmarkParseMedium/copy-32                   515.25       558.21       1.08x
BenchmarkParseMedium/nocopy-32                 607.90       716.11       1.18x
BenchmarkParseMedium/nocopy-par-32             9633.42      13078.09     1.36x
BenchmarkParseLarge/copy-32                    490.15       476.92       0.97x
BenchmarkParseLarge/nocopy-32                  605.24       598.32       0.99x
BenchmarkParseLarge/nocopy-par-32              10942.90     11134.47     1.02x
BenchmarkParseApache_builds/copy-32            930.14       966.07       1.04x
BenchmarkParseApache_builds/nocopy-32          1251.68      1334.90      1.07x
BenchmarkParseApache_builds/nocopy-par-32      18231.76     18459.40     1.01x
BenchmarkParseCanada/copy-32                   189.39       185.39       0.98x
BenchmarkParseCanada/nocopy-32                 190.88       185.51       0.97x
BenchmarkParseCanada/nocopy-par-32             3279.35      3507.28      1.07x
BenchmarkParseCitm_catalog/copy-32             1024.47      1148.50      1.12x
BenchmarkParseCitm_catalog/nocopy-32           1380.69      1316.56      0.95x
BenchmarkParseCitm_catalog/nocopy-par-32       19351.41     18901.14     0.98x
BenchmarkParseGithub_events/copy-32            932.56       895.44       0.96x
BenchmarkParseGithub_events/nocopy-32          1148.61      1145.09      1.00x
BenchmarkParseGithub_events/nocopy-par-32      20218.27     20569.28     1.02x
BenchmarkParseGsoc_2018/copy-32                2583.76      2604.32      1.01x
BenchmarkParseGsoc_2018/nocopy-32              3277.39      3281.82      1.00x
BenchmarkParseGsoc_2018/nocopy-par-32          23955.43     23653.54     0.99x
BenchmarkParseInstruments/copy-32              714.89       724.99       1.01x
BenchmarkParseInstruments/nocopy-32            834.47       849.09       1.02x
BenchmarkParseInstruments/nocopy-par-32        13055.99     13068.51     1.00x
BenchmarkParseMarine_ik/copy-32                226.61       229.36       1.01x
BenchmarkParseMarine_ik/nocopy-32              235.20       236.04       1.00x
BenchmarkParseMarine_ik/nocopy-par-32          4320.37      4548.98      1.05x
BenchmarkParseMesh/copy-32                     188.76       192.25       1.02x
BenchmarkParseMesh/nocopy-32                   191.64       192.94       1.01x
BenchmarkParseMesh/nocopy-par-32               3529.77      3695.56      1.05x
BenchmarkParseMesh_pretty/copy-32              371.04       359.67       0.97x
BenchmarkParseMesh_pretty/nocopy-32            372.87       360.12       0.97x
BenchmarkParseMesh_pretty/nocopy-par-32        6552.88      6654.70      1.02x
BenchmarkParseNumbers/copy-32                  205.10       212.15       1.03x
BenchmarkParseNumbers/nocopy-32                205.00       212.47       1.04x
BenchmarkParseNumbers/nocopy-par-32            3660.71      3762.58      1.03x
BenchmarkParseRandom/copy-32                   482.29       478.80       0.99x
BenchmarkParseRandom/nocopy-32                 698.53       675.69       0.97x
BenchmarkParseRandom/nocopy-par-32             10835.45     10690.44     0.99x
BenchmarkParseTwitter/copy-32                  1004.00      999.95       1.00x
BenchmarkParseTwitter/nocopy-32                1341.20      1347.55      1.00x
BenchmarkParseTwitter/nocopy-par-32            18379.99     18498.35     1.01x
BenchmarkParseTwitterEscaped/copy-32           560.39       558.86       1.00x
BenchmarkParseTwitterEscaped/nocopy-32         647.09       648.88       1.00x
BenchmarkParseTwitterEscaped/nocopy-par-32     11495.89     11602.21     1.01x
BenchmarkParseUpdate_center/copy-32            717.60       698.93       0.97x
BenchmarkParseUpdate_center/nocopy-32          1042.76      1018.24      0.98x
BenchmarkParseUpdate_center/nocopy-par-32      14683.23     14599.63     0.99x
BenchmarkJsonParserLarge/nocopy-32             514.91       478.97       0.93x

benchmark                                      old allocs     new allocs     delta
BenchmarkParseSmall/copy-32                    11             1              -90.91%
BenchmarkParseSmall/nocopy-32                  11             1              -90.91%
BenchmarkParseSmall/nocopy-par-32              10             0              -100.00%
BenchmarkParseMedium/copy-32                   11             1              -90.91%
BenchmarkParseMedium/nocopy-32                 11             1              -90.91%
BenchmarkParseMedium/nocopy-par-32             10             0              -100.00%
BenchmarkParseLarge/copy-32                    16             3              -81.25%
BenchmarkParseLarge/nocopy-32                  16             3              -81.25%
BenchmarkParseLarge/nocopy-par-32              15             2              -86.67%
BenchmarkParseApache_builds/copy-32            21             3              -85.71%
BenchmarkParseApache_builds/nocopy-32          21             3              -85.71%
BenchmarkParseApache_builds/nocopy-par-32      20             2              -90.00%
BenchmarkParseCanada/copy-32                   249            3              -98.80%
BenchmarkParseCanada/nocopy-32                 249            3              -98.80%
BenchmarkParseCanada/nocopy-par-32             249            3              -98.80%
BenchmarkParseCitm_catalog/copy-32             109            3              -97.25%
BenchmarkParseCitm_catalog/nocopy-32           109            3              -97.25%
BenchmarkParseCitm_catalog/nocopy-par-32       109            3              -97.25%
BenchmarkParseGithub_events/copy-32            16             3              -81.25%
BenchmarkParseGithub_events/nocopy-32          16             3              -81.25%
BenchmarkParseGithub_events/nocopy-par-32      15             2              -86.67%
BenchmarkParseGsoc_2018/copy-32                66             3              -95.45%
BenchmarkParseGsoc_2018/nocopy-32              66             3              -95.45%
BenchmarkParseGsoc_2018/nocopy-par-32          66             3              -95.45%
BenchmarkParseInstruments/copy-32              32             3              -90.62%
BenchmarkParseInstruments/nocopy-32            32             3              -90.62%
BenchmarkParseInstruments/nocopy-par-32        31             2              -93.55%
BenchmarkParseMarine_ik/copy-32                466            3              -99.36%
BenchmarkParseMarine_ik/nocopy-32              466            3              -99.36%
BenchmarkParseMarine_ik/nocopy-par-32          466            3              -99.36%
BenchmarkParseMesh/copy-32                     121            3              -97.52%
BenchmarkParseMesh/nocopy-32                   121            3              -97.52%
BenchmarkParseMesh/nocopy-par-32               120            3              -97.50%
BenchmarkParseMesh_pretty/copy-32              121            3              -97.52%
BenchmarkParseMesh_pretty/nocopy-32            121            3              -97.52%
BenchmarkParseMesh_pretty/nocopy-par-32        120            3              -97.50%
BenchmarkParseNumbers/copy-32                  27             3              -88.89%
BenchmarkParseNumbers/nocopy-32                27             3              -88.89%
BenchmarkParseNumbers/nocopy-par-32            26             2              -92.31%
BenchmarkParseRandom/copy-32                   75             3              -96.00%
BenchmarkParseRandom/nocopy-32                 75             3              -96.00%
BenchmarkParseRandom/nocopy-par-32             74             2              -97.30%
BenchmarkParseTwitter/copy-32                  52             3              -94.23%
BenchmarkParseTwitter/nocopy-32                52             3              -94.23%
BenchmarkParseTwitter/nocopy-par-32            51             2              -96.08%
BenchmarkParseTwitterEscaped/copy-32           52             3              -94.23%
BenchmarkParseTwitterEscaped/nocopy-32         52             3              -94.23%
BenchmarkParseTwitterEscaped/nocopy-par-32     51             2              -96.08%
BenchmarkParseUpdate_center/copy-32            57             3              -94.74%
BenchmarkParseUpdate_center/nocopy-32          57             3              -94.74%
BenchmarkParseUpdate_center/nocopy-par-32      56             2              -96.43%
BenchmarkJsonParserLarge/nocopy-32             50             37             -26.00%

benchmark                                      old bytes     new bytes     delta
BenchmarkParseSmall/copy-32                    664           16            -97.59%
BenchmarkParseSmall/nocopy-32                  664           16            -97.59%
BenchmarkParseSmall/nocopy-par-32              660           15            -97.73%
BenchmarkParseMedium/copy-32                   664           16            -97.59%
BenchmarkParseMedium/nocopy-32                 664           16            -97.59%
BenchmarkParseMedium/nocopy-par-32             663           15            -97.74%
BenchmarkParseLarge/copy-32                    795           76            -90.44%
BenchmarkParseLarge/nocopy-32                  792           72            -90.91%
BenchmarkParseLarge/nocopy-par-32              783           64            -91.83%
BenchmarkParseApache_builds/copy-32            956           117           -87.76%
BenchmarkParseApache_builds/nocopy-32          931           86            -90.76%
BenchmarkParseApache_builds/nocopy-par-32      901           65            -92.79%
BenchmarkParseCanada/copy-32                   37406         32415         -13.34%
BenchmarkParseCanada/nocopy-32                 37400         30787         -17.68%
BenchmarkParseCanada/nocopy-par-32             39985         34793         -12.98%
BenchmarkParseCitm_catalog/copy-32             6912          3515          -49.15%
BenchmarkParseCitm_catalog/nocopy-32           5501          2652          -51.79%
BenchmarkParseCitm_catalog/nocopy-par-32       6289          3334          -46.99%
BenchmarkParseGithub_events/copy-32            801           82            -89.76%
BenchmarkParseGithub_events/nocopy-32          793           73            -90.79%
BenchmarkParseGithub_events/nocopy-par-32      783           64            -91.83%
BenchmarkParseGsoc_2018/copy-32                17884         15690         -12.27%
BenchmarkParseGsoc_2018/nocopy-32              9712          7812          -19.56%
BenchmarkParseGsoc_2018/nocopy-par-32          30098         30619         +1.73%
BenchmarkParseInstruments/copy-32              1308          206           -84.25%
BenchmarkParseInstruments/nocopy-32            1260          151           -88.02%
BenchmarkParseInstruments/nocopy-par-32        1170          69            -94.10%
BenchmarkParseMarine_ik/copy-32                173465        161947        -6.64%
BenchmarkParseMarine_ik/nocopy-32              177267        163826        -7.58%
BenchmarkParseMarine_ik/nocopy-par-32          249116        228206        -8.39%
BenchmarkParseMesh/copy-32                     14406         11167         -22.48%
BenchmarkParseMesh/nocopy-32                   14440         11062         -23.39%
BenchmarkParseMesh/nocopy-par-32               18010         12177         -32.39%
BenchmarkParseMesh_pretty/copy-32              11088         8086          -27.07%
BenchmarkParseMesh_pretty/nocopy-32            10978         8022          -26.93%
BenchmarkParseMesh_pretty/nocopy-par-32        11616         8249          -28.99%
BenchmarkParseNumbers/copy-32                  1235          248           -79.92%
BenchmarkParseNumbers/nocopy-32                1235          245           -80.16%
BenchmarkParseNumbers/nocopy-par-32            1048          73            -93.03%
BenchmarkParseRandom/copy-32                   4165          2137          -48.69%
BenchmarkParseRandom/nocopy-32                 3141          1032          -67.14%
BenchmarkParseRandom/nocopy-par-32             2467          589           -76.12%
BenchmarkParseTwitter/copy-32                  2623          1058          -59.66%
BenchmarkParseTwitter/nocopy-32                2014          432           -78.55%
BenchmarkParseTwitter/nocopy-par-32            1668          90            -94.60%
BenchmarkParseTwitterEscaped/copy-32           3028          1444          -52.31%
BenchmarkParseTwitterEscaped/nocopy-32         2509          929           -62.97%
BenchmarkParseTwitterEscaped/nocopy-par-32     1728          159           -90.80%
BenchmarkParseUpdate_center/copy-32            3337          1664          -50.13%
BenchmarkParseUpdate_center/nocopy-32          2114          417           -80.27%
BenchmarkParseUpdate_center/nocopy-par-32      1774          88            -95.04%
BenchmarkJsonParserLarge/nocopy-32             1353          634           -53.14%