Remove simd bitpacking implementations for high bit width

quickwit-oss / bitpacking

SIMD algorithms for integer compression via bitpacking. This crate is a port of a C library called simdcomp.

MIT License

272 stars 30 forks source link

Remove simd bitpacking implementations for high bit width #47

Open fulmicoton opened 1 year ago

fulmicoton commented 1 year ago

For sorted stuff, having a super fast implementation is not necessary above bitwidth = 8. We can fallback to the scalar implementation. For unsorted stuff, I think we can also restrict ourselves to bitwidth <= 8, although this is a trade-off.

PSeitz commented 11 months ago

The amount of generated code seems excessive currently

➜  bitpacking git:(master) cargo expand | wc -l

330128

fulmicoton commented 11 months ago

Definitely :)