elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
108 stars 4.93k forks source link

[8.x](backport #41584) [libbeat] Go Benchmarks comparing compress/gzip and klauspost/compress #41669

Closed mergify[bot] closed 3 days ago

mergify[bot] commented 3 days ago

Results

We send data to the _bulk endpoint and benchmark the results with and without the https://github.com/klauspost/compress library.

Results with varying compression level and event size

Time per operation

Test without_klauspost.txt with_klauspost.txt Improvement
Publish/5_events_with_compression_level_1-12 91.52µ ± 11% 76.34µ ± 2% -16.58%
Publish/5_events_with_compression_level_4-12 126.78µ ± 8% 82.20µ ± 0% -35.16%
Publish/5_events_with_compression_level_7-12 118.2µ ± 1% 109.5µ ± 0% -7.34%
Publish/5_events_with_compression_level_9-12 131.1µ ± 8% 147.3µ ± 3% +12.36%
Publish/50_events_with_compression_level_1-12 453.8µ ± 5% 347.3µ ± 2% -23.47%
Publish/50_events_with_compression_level_4-12 676.3µ ± 5% 422.7µ ± 2% -37.50%
Publish/50_events_with_compression_level_7-12 704.8µ ± 2% 592.2µ ± 1% -15.98%
Publish/50_events_with_compression_level_9-12 878.8µ ± 3% 991.6µ ± 2% +12.84%
Publish/500_events_with_compression_level_1-12 435.4µ ± 1% 343.6µ ± 2% -21.09%
Publish/500_events_with_compression_level_4-12 636.7µ ± 1% 428.6µ ± 12% -32.68%
Publish/500_events_with_compression_level_7-12 681.7µ ± 1% 601.5µ ± 1% -11.77%
Publish/500_events_with_compression_level_9-12 855.8µ ± 2% 992.4µ ± 2% +15.96%
Geomean 364.7µ 309.3µ -15.17%

Bytes per operation

Test without_klauspost.txt with_klauspost.txt Improvement
Publish/5_events_with_compression_level_1-12 18.78Ki ± 0% 18.74Ki ± 0% -0.20%
Publish/5_events_with_compression_level_4-12 18.83Ki ± 0% 18.68Ki ± 0% -0.79%
Publish/5_events_with_compression_level_7-12 18.83Ki ± 0% 18.75Ki ± 0% -0.41%
Publish/5_events_with_compression_level_9-12 18.79Ki ± 0% 18.82Ki ± 0% ~
Publish/50_events_with_compression_level_1-12 88.73Ki ± 0% 88.46Ki ± 0% -0.31%
Publish/50_events_with_compression_level_4-12 88.27Ki ± 0% 88.32Ki ± 0% ~
Publish/50_events_with_compression_level_7-12 88.31Ki ± 0% 88.51Ki ± 0% +0.23%
Publish/50_events_with_compression_level_9-12 88.37Ki ± 0% 88.83Ki ± 0% +0.52%
Publish/500_events_with_compression_level_1-12 88.39Ki ± 0% 88.44Ki ± 0% ~
Publish/500_events_with_compression_level_4-12 88.30Ki ± 0% 88.47Ki ± 0% +0.19%
Publish/500_events_with_compression_level_7-12 88.28Ki ± 0% 88.55Ki ± 0% +0.31%
Publish/500_events_with_compression_level_9-12 88.27Ki ± 0% 88.86Ki ± 0% +0.66%
Geomean 52.76Ki 52.78Ki +0.04%

Summary

With the ' klauspost ' library, there is a significant speedup across tests for most compression levels with varying event sizes. But with compression level 9, it is slower - resulting in a performance drop of around 12-16%.

This PR also replaces compress/gzip to klauspost/compress/gzip

Checklist

Related issues


This is an automatic backport of pull request #41584 done by Mergify.

botelastic[bot] commented 3 days ago

This pull request doesn't have a Team:<team> label.