Facebook has gotten into the LZ variant codec game, too. Its zstd algorithm allows for a wide band of tradeoff in space vs time. It has seen uptake my Linux distributions for package compression as well as in storage engines like RocksDB (Facebook's much-improved fork of Google's LevelDB). Bulk_extractor should recognize and decode zstd data for recursive processing. A BSD-licensed reference implementation is available.
Facebook has gotten into the LZ variant codec game, too. Its
zstdalgorithm allows for a wide band of tradeoff in space vs time. It has seen uptake my Linux distributions for package compression as well as in storage engines like RocksDB (Facebook's much-improved fork of Google's LevelDB). Bulk_extractor should recognize and decode zstd data for recursive processing. A BSD-licensed reference implementation is available.https://facebook.github.io/zstd/