search

lz4 / lz4-java

LZ4 compression for Java
Apache License 2.0
1.1k stars 253 forks source link

Update to lz4 1.9.0 for 12-18% decompression improvement #138

Closed ijuma closed 5 years ago

ijuma commented 5 years ago
Version v1.8.3 v1.9.0 Improvement
enwik8 4090 MB/s 4560 MB/s +12%
calgary.tar 4320 MB/s 4860 MB/s +13%
silesia.tar 4210 MB/s 4970 MB/s +18%

https://github.com/lz4/lz4/releases/tag/v1.9.0

odaira commented 5 years ago

Updated to lz4 1.9.0 in master. e8076d1f221b56fdec166f731be98113d0b03835 Will release the next version based on this.

ijuma commented 5 years ago

Thanks!

ijuma commented 5 years ago

@odaira do you have a timeline for the next release? I was hoping to include it in the next Kafka version due in a few weeks.

odaira commented 5 years ago

OK, I'll work on this item first. Let me set the target to May 10.

ijuma commented 5 years ago

Thanks!

patric42 commented 5 years ago

Just a heads up: Release 1.91 fixed a bug "decompression functions were reading a few bytes beyond input size", which seems somewhat serious.

ijuma commented 5 years ago

Yeah, that was integrated in https://github.com/lz4/lz4-java/commit/c5ae694a3aa8b33ef87fd0486727a7d1e8f71367

patric42 commented 5 years ago

Oh, great - didn't see that. Thanks!

odaira commented 5 years ago

As another heads-up, because LZ4_decompress_fast() got deprecated, I'll add @Deprecated to LZ4FastDecompressor. LZ4BlockInputStream will also need to be modified to not use LZ4FastDecompressor, but that's a work item after 1.6.0.

odaira commented 5 years ago

Released 1.6.0.

ijuma commented 5 years ago

Great!