Open VladRodionov opened 10 years ago
Done. Added StorageOptimizer threads, can configure # of threads and optimization level: 1-5
1 - It recompresses to LZ4HC all objects 2 - 30% (least recently used) to GZ, other to LZ4HC 3 - 50%-50% GZ/LZ4HC 4 - 75% to GZ 5 - 100% to GZ
This is probably not for the first release, as since test StorageOptimizerTest fail when using put with codec method, but succeed when using regular put (see code).
Another thing: this will work only for block cache, as since row cache uses API which has not been converted yet to multi-codec.
Idea is to use the fastest compression when object is being placed into cache and later re-compress it using more advanced version (LZ4 =>LZ4-HC).
Some optimizations: https://groups.google.com/forum/#!topic/lz4c/fdlAfZzSAr4