search

pierrec / node-lz4

LZ4 fast compression algorithm for NodeJS
MIT License
438 stars 98 forks source link

Considerably slower than Snappy #114

Closed Awendel closed 2 years ago

Awendel commented 2 years ago

LZ4 on paper is meant to beat Snappy, especially with decompression.

Yet comparing this library against https://github.com/Brooooooklyn/snappy, it really is a lot slower, it doesn't even come close.

Have there been any benchmarks / tests pinpointing to why this package is so much slower?

ankon commented 2 years ago

Out of interest: Could you expand a bit on how measured?

Awendel commented 2 years ago

Sure, was considering it as compression algo for Redis DB.

Had large Array of JSON Objects (each a few kb) and then looped through, compressing & decompressing respectively and adding up the latencies. Snappy did that in like 90ms whereas node-lz4 took like around 300ms

pierrec commented 2 years ago

Not sure. Benchmarks are always fussy and you should choose the algorithm based on your findings. Note that LZ4 is not very good (as in fast) for small inputs. It really shines on large ones. YMMV