Add ISA-L to accelerate gzip compression in case of high network load

[giantcroc]

Title: Add ISA-L to accelerate gzip compression in case of high network load

Description: ISA-L is a collection of optimized low-level functions targeting storage applications. The low-level functions of ISA-L are written in manual assembly code that many details are tuned to achieve optimal performance.

ISA-L uses efficient SIMD (Single Instruction, Multiple Data) instructions like AVX512 to maximize the use of CPU micro-architecture to accelerate the calculation process of compression algorithms. And ISA-L mainly optimized the implementation of CRC, DEFLATE and Huffman encoding for gzip compression algorithm.

We have performed a benchmark that test the compression performance under different data scales with ISA-L and Zlib, see figure in bellow. Specifically, we used compressor_speed_test tool to do benchmark, and averaged results of all compression levels for drawing the graph. And you can see all benchmark logs in isa-l-benchmark.

As shown in the figure above, we believe that ISA-L can help Envoy to improve gzip compression performance in case of high network load, and we are working on integrating ISA-L into Envoy.

[optional Relevant Links:] ISA-L Igzip: Improvements to a Fast Deflate ISA-L tool ISA-L videos ISA-L intro ISA-L in DPDK

[giantcroc]

@zuercher Hi, zuercher! I find that you added an area/extension label. Do you think we could add ISA-L to source/extensions?

[zuercher]

This issue relates to an Envoy extension (the gzip filter) regardless of where the code lives, thus area/extension.

The extension policy and dependency policy documents describe the conditions under which such a change would be accepted. Where the code would reside in the repository is a function of that. I recommend you review those documents first.

[giantcroc]

OK, thanks for your advice!

This issue relates to an Envoy extension (the gzip filter) regardless of where the code lives, thus area/extension.

The extension policy and dependency policy documents describe the conditions under which such a change would be accepted. Where the code would reside in the repository is a function of that. I recommend you review those documents first.

OK, thanks for your advice!

[rainingmaster]

Nice job! Here is Jinhua from ByteDance, and I am the code owner for /*/extensions/compression/zstd part. It is my pleasure to to sponsor this enhancement.

Since ByteDance's traffic needs to be compressed, we've been working with the Intel to offload that pressure to the hardware.

[qiming-007]

Thanks @rainingmaster for sponsoring the feature.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.