boostorg / histogram

Fast multi-dimensional generalized histogram with convenient interface for C++14
Boost Software License 1.0
313 stars 74 forks source link

Implemented a memory-compressed storage #318

Open HDembinski opened 3 years ago

HDembinski commented 3 years ago

Sophisticated analyses use high-dimensional histograms, both disc and memory usage can become an issue. Disc usage is easy to fix by writing a compressed stream with Boost.Serialization and Boost.IOStreams. But this does not fix the memory issue.

It would be interesting to add an experimental memory-compressed storage and see how that performs. In-memory compression can be achieved in several ways.

HDembinski commented 2 years ago

BLOSC is BSD licensed, so compatible with BSL.

https://www.blosc.org/pages/blosc-in-depth/ https://github.com/Blosc/c-blosc2