search

apache / distributedlog

Apache DistributedLog
Apache License 2.0
185 stars 74 forks source link

Issue 157: Introduce JMH for micro benchmarking #158

Closed zhaijack closed 7 years ago

zhaijack commented 7 years ago

Descriptions of the changes in this PR:

zhaijack commented 7 years ago

This is the pull request for #157

sijie commented 7 years ago

/cc @leighst

zhaijack commented 7 years ago

The benchmark result for latest master:

Benchmark                                                                 (size)   Mode  Cnt       Score      Error   Units
CompressionBenchmark.testCompressLZ4                                          10  thrpt   50    2422.408 ±   44.161  ops/ms
CompressionBenchmark.testCompressLZ4                                         100  thrpt   50    1793.394 ±   27.825  ops/ms
CompressionBenchmark.testCompressLZ4                                        1000  thrpt   50     928.596 ±   14.488  ops/ms
CompressionBenchmark.testCompressLZ4                                       10000  thrpt   50     239.443 ±    4.752  ops/ms
CompressionBenchmark.testCompressNone                                         10  thrpt   50  207570.657 ± 4656.234  ops/ms
CompressionBenchmark.testCompressNone                                        100  thrpt   50   90762.739 ± 1753.916  ops/ms
CompressionBenchmark.testCompressNone                                       1000  thrpt   50   10532.599 ±  207.739  ops/ms
CompressionBenchmark.testCompressNone                                      10000  thrpt   50    1034.792 ±   20.067  ops/ms
CompressionBenchmark.testDecompressLZ4                                        10  thrpt   50    4265.879 ±   80.825  ops/ms
CompressionBenchmark.testDecompressLZ4                                       100  thrpt   50    3975.214 ±  130.441  ops/ms
CompressionBenchmark.testDecompressLZ4                                      1000  thrpt   50    1821.820 ±   37.923  ops/ms
CompressionBenchmark.testDecompressLZ4                                     10000  thrpt   50     298.692 ±    8.591  ops/ms
CompressionBenchmark.testDecompressNone                                       10  thrpt   50  202194.272 ± 4431.166  ops/ms
CompressionBenchmark.testDecompressNone                                      100  thrpt   50   80907.829 ± 3869.202  ops/ms
CompressionBenchmark.testDecompressNone                                     1000  thrpt   50   10041.016 ±  391.215  ops/ms
CompressionBenchmark.testDecompressNone                                    10000  thrpt   50    1016.756 ±   26.645  ops/ms

The jvm allocation rate is huge, even with compression none:

CompressionBenchmark.testCompressLZ4:·gc.alloc.rate                           10  thrpt   50     123.090 ±    2.254  MB/sec
CompressionBenchmark.testCompressLZ4:·gc.alloc.rate                          100  thrpt   50     291.617 ±    4.542  MB/sec
CompressionBenchmark.testCompressLZ4:·gc.alloc.rate                         1000  thrpt   50    1217.140 ±   19.148  MB/sec
CompressionBenchmark.testCompressLZ4:·gc.alloc.rate                        10000  thrpt   50    3061.999 ±   60.421  MB/sec
CompressionBenchmark.testCompressNone:·gc.alloc.rate                          10  thrpt   50    4219.536 ±   95.041  MB/sec
CompressionBenchmark.testCompressNone:·gc.alloc.rate                         100  thrpt   50    6918.226 ±  134.556  MB/sec
CompressionBenchmark.testCompressNone:·gc.alloc.rate                        1000  thrpt   50    6796.270 ±  133.545  MB/sec
CompressionBenchmark.testCompressNone:·gc.alloc.rate                       10000  thrpt   50    6581.972 ±  126.883  MB/sec
CompressionBenchmark.testDecompressLZ4:·gc.alloc.rate                         10  thrpt   50     238.471 ±    4.534  MB/sec
CompressionBenchmark.testDecompressLZ4:·gc.alloc.rate                        100  thrpt   50    1131.557 ±   37.133  MB/sec
CompressionBenchmark.testDecompressLZ4:·gc.alloc.rate                       1000  thrpt   50    4683.743 ±   97.727  MB/sec
CompressionBenchmark.testDecompressLZ4:·gc.alloc.rate                      10000  thrpt   50    7619.130 ±  218.422  MB/sec
CompressionBenchmark.testDecompressNone:·gc.alloc.rate                        10  thrpt   50    4108.263 ±   90.288  MB/sec
CompressionBenchmark.testDecompressNone:·gc.alloc.rate                       100  thrpt   50    6168.327 ±  295.150  MB/sec
CompressionBenchmark.testDecompressNone:·gc.alloc.rate                      1000  thrpt   50    6481.412 ±  252.577  MB/sec
CompressionBenchmark.testDecompressNone:·gc.alloc.rate                     10000  thrpt   50    6467.379 ±  169.089  MB/sec
zhaijack commented 7 years ago

pushed a new change to exclude generated files for findbugs:check

zhaijack commented 7 years ago

the test result from 0.4.0-incubating (similar result as current master, since there is any change to compression codec)

expected to see performance improvements at #151 

Benchmark                                                                 (size)   Mode  Cnt       Score      Error   Units
CompressionBenchmark.testCompressLZ4                                          10  thrpt   50    2422.408 ±   44.161  ops/ms
CompressionBenchmark.testCompressLZ4                                         100  thrpt   50    1793.394 ±   27.825  ops/ms
CompressionBenchmark.testCompressLZ4                                        1000  thrpt   50     928.596 ±   14.488  ops/ms
CompressionBenchmark.testCompressLZ4                                       10000  thrpt   50     239.443 ±    4.752  ops/ms
CompressionBenchmark.testCompressNone                                         10  thrpt   50  207570.657 ± 4656.234  ops/ms
CompressionBenchmark.testCompressNone                                        100  thrpt   50   90762.739 ± 1753.916  ops/ms
CompressionBenchmark.testCompressNone                                       1000  thrpt   50   10532.599 ±  207.739  ops/ms
CompressionBenchmark.testCompressNone                                      10000  thrpt   50    1034.792 ±   20.067  ops/ms
CompressionBenchmark.testDecompressLZ4                                        10  thrpt   50    4265.879 ±   80.825  ops/ms
CompressionBenchmark.testDecompressLZ4                                       100  thrpt   50    3975.214 ±  130.441  ops/ms
CompressionBenchmark.testDecompressLZ4                                      1000  thrpt   50    1821.820 ±   37.923  ops/ms
CompressionBenchmark.testDecompressLZ4                                     10000  thrpt   50     298.692 ±    8.591  ops/ms
CompressionBenchmark.testDecompressNone                                       10  thrpt   50  202194.272 ± 4431.166  ops/ms
CompressionBenchmark.testDecompressNone                                      100  thrpt   50   80907.829 ± 3869.202  ops/ms
CompressionBenchmark.testDecompressNone                                     1000  thrpt   50   10041.016 ±  391.215  ops/ms
CompressionBenchmark.testDecompressNone                                    10000  thrpt   50    1016.756 ±   26.645  ops/ms
sijie commented 7 years ago

all CI job passed.