search

lemire / FastPFor

The FastPFOR C++ library: Fast integer compression
Apache License 2.0
883 stars 124 forks source link

Point Based Access? #75

Closed linuslh1996 closed 4 years ago

linuslh1996 commented 4 years ago

Hey, I wanted to ask if this library supports point-based access? With Point-Based Access I mean something like this:

uint_32t single_decompressed_value = decoded.get(i); // only the number at the i-th position gets decompressed

If this is not the case: Can you maybe give an insight into what your considerations are in this area? And since you seem to be quite the experts, would you know of other good libraries that support it?

lemire commented 4 years ago

The following paper covers point queries in a format called Stream VByte...

Stream VByte: Faster Byte-Oriented Integer Compression, Information Processing Letters 130, February 2018, Pages 1-6 https://arxiv.org/pdf/1709.08990.pdf

There is a more general paper...

Efficient Integer-Key Compression in a Key-Value Store using SIMD Instructions Information Systems 66, 2017 https://arxiv.org/abs/1611.05428

You will find many of the implementations in the following library: https://github.com/lemire/SIMDCompressionAndIntersection

I am going to close this issue. Reopen a new issue if this did not answer your question.