Slower than Snappy for smaller sizes.

pierrec / lz4

LZ4 compression and decompression in pure Go

BSD 3-Clause "New" or "Revised" License

879 stars 142 forks source link

Closed jeschkies closed 7 months ago

jeschkies commented 7 months ago

I ran some small benchmarks to see how LZ4 would compare to Snappy on smaller datasets below 16MiB (https://github.com/jeschkies/loki/blob/karsten/investigate-columnar-chunks-enc/pkg/chunkenc/batch_test.go#L90) and found that LZ4 was roughly 40% slower than Snappy. It outperformed Snappy dramatically on datasets in the hundreds of megabytes. Is this to be expected? I'm suspecting I'm missing something.

jeschkies commented 7 months ago

Nevermind. This was for compression :joy: