search

powturbo / TurboPFor-Integer-Compression

Fastest Integer Compression
GNU General Public License v2.0
766 stars 113 forks source link

Integer PFor and bitpack zigzag of delta? #41

Closed dsentinel closed 5 years ago

powturbo commented 5 years ago

Is it possible to bit pack a zigzag of delta's?

use icapp to find out the best TurboPFor function for your data. You can use the float functions also for integers and vice versa. You can also use the float functions for double content or 16 bits functions for 32 bits contents.... There are some cases where you get better compression.

Have you looked at using XOR instead of or like delta's?

In general zigzag/delta compress better than xor.

What is your bitpacking algorithm?

see the references at the bottom of the readme

powturbo commented 5 years ago

Is the fpzzenc32 lossy?

All TurboPFor functions are lossless. There are two preprocessing transforms padfloat + paddouble that can be used to convert float or double reducing the precision.

Which one?

SIMD Compression and the Intersection of Sorted Integers

powturbo commented 5 years ago

Hi, it will be nice, if you can post your pull request for clang-700.1.76 mac os 10.10.5 with the current TurboPFor version. I have no possibility for testing TurboPFor under a mac os.