search

tspurway / hustle

A column oriented, embarrassingly distributed relational event database.
Other
240 stars 36 forks source link

Conider using Blosc #52

Open esc opened 10 years ago

esc commented 10 years ago

I saw that you are using lz4 for compression. Perhaps you are also interested in the Blosc metacodec:

http://www.blosc.org

It actually supports using lz4 under the hood and can yield speed and ratio improvements over plain lz4 on certain kinds of data (e.g. timeseries) due to multithreading and pre-conditioning.

We have some pretty well tuned python bindings too:

https://github.com/Blosc/python-blosc

ncloudioj commented 10 years ago

Thanks for letting us know this library, it sounds pretty promising. We'll definitely give it a shot.

esc commented 10 years ago

Cool! If you need any more infos or have questions, feel free to contact the goggle group or contact me personally.