search

powturbo / TurboPFor-Integer-Compression

Fastest Integer Compression
GNU General Public License v2.0
766 stars 113 forks source link

256w vs. 256v encoder/decoder #86

Closed mayeranalytics closed 1 year ago

mayeranalytics commented 2 years ago

What is the difference between p4nenc256w32 and p4nenc256v32? The v stands for vertical, I believe. I cannot find any information about what 'vertical' means, nor what the 'w' variant might be. Looking at the decoder, instead of a corresponding decoder p4ndec256w32 there is p4nddec256w32 ('n' vs. 'nd'). Could you please document what these function are and how it makes sense that the 'w' encoder and 'w' decoder do not quite mirror each other? Thanks.

powturbo commented 1 year ago

p4nenc256w32 is like p4nenc256v32 but using sse 2x128 (128 bits SIMD) instead of AVX2 (256 bits). vertical means vertical bit packing, there is also horizontal bitpacking (not in TurboPFor). n = unlimited length. Functions witout 'n' are low level and process only one SIMD register. d = delta (non decreasing arrays), x = xor, z=zigzag delta (all values)