Closed ptaoussanis closed 10 years ago
Another interesting option could be lz4 (even though it's not high-compression).
http://code.google.com/p/lz4/ http://blog.jpountz.net/post/28092106032/wow-lz4-is-fast
That does look interesting. Am definitely open to other fast compressors though, practically speaking, I think there's probably more real-world application for some high compression stuff (esp. since we do already have Snappy).
Those are impressive numbers though, wow.
Yup I agree. It's really curiosity at this point, I don't know enough about lz4 to judge it really.
Just happened across one candidate for a high-ratio compressor based on LZMA2 and with a reference pure-Java implementation that beats (among others) bzip2 and gzip: Lib page: http://tukaani.org/xz/ Lib API docs: http://tukaani.org/xz/xz-javadoc/ Maven: http://mvnrepository.com/artifact/org.tukaani/xz Algo/container info: http://en.wikipedia.org/wiki/Xz Benchmark: http://stephane.lesimple.fr/blog/2010-07-20/lzop-vs-compress-vs-gzip-vs-bzip2-vs-lzma-vs-lzma2xz-benchmark-reloaded.html
Quick heads-up that I've just run into a need for this - going to experiment with getting an impl in today.
Pushed as v2.4.0
.
Snappy (currently implemented) offers a sensible default, but there are a lot of cases where a higher compression ratio for higher CPU usage would be a good tradeoff.
As of v2, adding new compressors is very simple (just implement the
ICompressor
protocol intaoensso.nippy.compression
).GZip would be a good start, and I'm sure there are some other higher-compression Java libs we can support.